JavaScript is not available.

for all of you asking how to keep in touch, post-twitterpocalypse: 1) i'm not actively abandoning ship 2) mipsytipsy@hachyderm.io...i don't really care for mastodon, but may experiment further 3) linkedin...but i decline spam invites, so add a comment that has the word "twitter"

Clay Dowling

@ClayDowling

I worked on a product that could ship hourly, and had to comply with European banking regulations about traceability. Your arguments about why you can't deploy on Friday are invalid. Listen to Charity.

Quote Tweet

Apr 15, 2019

If you're scared of pushing to production on Fridays, I recommend reassigning all your developer cycles off of feature development and onto your CI/CD process and observability tooling for as long as it takes to

fix that

Show this thread

Dec 20

At least ten people have apologized bc they couldn't send a comment with their connection request, so here is your useless LI fact of the day. After you send a connection request to someone, it presents an option to send a message with your invite. 📩AFTER.📨 🤷‍♀️

Quote Tweet

drewblas

@drewblas

Replying to @mipsytipsy

So just learned on accident that clicking connect on LinkedIn mobile app doesn’t give you an option to add a comment to the request. Also no way to undo. Sorry was not trying to spam you!

Replying to

and

I recently read somewhere that SRE is short for Sidecar Restart Engineer and this is my favourite SRE fact of the year 😋

Paul Johnston - current focus: moving electrons

Dec 20

The most important job your instrumentation can do is not to reconstruct process execution. The most important things are: 1) help you find it (every request is a needle in the haystack) 2) help you figure out what's different about the req you care about vs the ones you don't

You have to instrument knowing that the vast majority of debugging will take place in aggregate, not sitting in front of a single process reading its debug output waiting for it to execute. Or grepping a log for it.

You have to instrument without needing to know whether your request will hop the network 0 times or 1000 times, because even if it isn't now, it might be someday. Logs that don't get this are trash.

But there are at least two massive cognitive shifts that need to happen -- non negotiably. 1) shift left/shift right/shift whatever -- your code doesn't work til it works in prod. 2) stop thinking of your request like a single process, or point-in-time logs as instrumentation.

I love this question. ☺️ I think MOST of the technical minutiae of instrumentation will be solved to the point where you can hand wave them away. OTel is a huge step forward, of course; eBPF will eventually be another. Hw will be cheap enough that we can capture profiling data.

Earlier today we were doing a Twitter space and the question came up: how much of what we currently ask engineers to understand about observability (to do it well) will eventually be abstracted away? In 10 years, will devs need to understand sampling? Spans? Counters,gauges etc?

Read this for a tiny horror story 😱 about what happens when developers build distributed systems, then go on logging like it's 1999.

Quote Tweet

@PaulDJohnston

Replying to @mipsytipsy and @plantpowerjames

I've seen it several times now when trying to help people understand how to build observability into serverless and distributed systems. Almost every time they build the application without thinking about logging and have an expectation that the logging will just "work".

Show this thread

Narasimhan Sridharan

@narasimhansridh

"Being a good engineer involves blocking out interruptions, focusing on learning and solving hard problems Being a good manager involves being available to your team and interruptible, even being interrupt-driven" -

120

PSA: if you wnat to speak in a twitter spaces, you must log in from your phone. from computer won't work. TIL!

Quote Tweet

Gonna have a Twitter hangout in 8 min with @jeanqasaur, special guest @ZachAMcCoy, topic: how to observability from wherever you are

Zach McCoy @zachmccoy@fosstodon.org

we are now trying to debug twitter spaces, which won't allow us to make

@jeanqasaur

a speaker 🙈

Quote Tweet

@ZachAMcCoy

We're live now with @mipsytipsy and @jeanqasaur talking about Prod-Care Routines for All! #monitoring #observability twitter.com/i/spaces/1ZkKz

Zach McCoy @zachmccoy@fosstodon.org

We're live now with

and

talking about Prod-Care Routines for All! #monitoring #observability

Gonna have a Twitter hangout in 8 min with

@jeanqasaur

, special guest

@ZachAMcCoy

, topic: how to observability from wherever you are 🐝🌈✨

shon

@inkedtater

Honeycomb has some killer tech for release cycle automation. You should check it out. They are #TaterOps approved

Quote Tweet

i LOVE this!!! OMG. "never any issues" is a pretty high bar though.. how about this?

Rule of Thumb:

~ Deploy Friday mornings when you can successfully deploy multiple times a week ~ Deploy Friday afternoon when you can successfully deploy hourly twitter.com/apocraphilia/s…

Show this thread

Kamil Szymański

@kszdev

Dec 10

The happiness and satisfaction of your users and your engineers tend to raise and fall in tandem -

Kamil Szymański

@kszdev

Dec 10

High-performing teams are the ones that: spend the majority of their time solving interesting, novel problems that move the business materially forward -

"[ADRs] are one of the secret sauce ingredients for having a remote org" 🙌

Quote Tweet

Howard van Rooijen

@HowardvRooijen

Dec 10

Replying to @mipsytipsy and @beddari

It's one of the secret sauce ingredients for having a remote org. I really need to update github.com/endjin/dotnet- there are some good templates in github.com/endjin/dotnet-

here's the talk i gave with

@circleci

on why honeycomb adopted graviton, and how circleci allowed us to test, maintain and deploy onto two architectures indefinitely with no manual shuffling.

Quote Tweet

Marcos Ortiz

@marcosluis2186

Dec 9

8/ [HIGHLY RECOMMENDED] PRT224: Overcoming migration hurdles with CircleCI and AWS Graviton, by @mipsytipsy (CTO, @honeycombio ), Tom Trahan ( @CircleCI ) and William Koplitz (@awscloud ) youtube.com/watch?v=Yl3sLn #reInvent2022

Show this thread

Vlad Ionescu (he/him)

@iamvlaaaaaaad

Dec 7

"And we rolled back. And by 'rolling back' I mean we flipped a feature flag cause we're not cavemen" -

🤣 YAS!

great thread of a great SRECon talk by

@istathar

, with lots of honeycomb screenshots and solid advice.

Quote Tweet

Christine Yen

@cyen

Dec 7

ooh, i love this: retries became really painful (and, eventually, useless to visualize)—so they went back to basics and just captured retries as a numeric "retry" counter as a separate field on the span/trace, instead.

Show this thread

if you weren't there for the terrific P99conf -- or even if you were -- this is an outstanding recap of some phenomenal talks. gets better every year!

Quote Tweet

ScyllaDB

@ScyllaDB

Dec 6

The @thenewstack examined the best practices and lessons learned that @bcantrill @lizrice @DorLaor @AviKivity @giltene @mitsuhiko and @mipsytipsy's shared during their #P99CONF keynotes. Check out their detailed insights now. bit.ly/3h1ANqD #ScyllaDB #Rustlang #Golang

and the C.H.A.R.I.T.Y system we used for monitoring and alerting 😂 (i have no recollection of what the letters stood for, lol)

Quote Tweet

Thomas Bouldin

@inlined

Dec 6

Replying to @puf and @anildash

We used to talk about how our “Charity” system ensured fairness across projects at Parse. At least we used to before people picked up that this was just @mipsytipsy’s name.

i take it back, this one is better 😂 "honeycomb: we turn booking a colonoscopy into having chocolate delivered to your home" nailed it 👌

Quote Tweet

Dec 4

Replying to @maggiepint

I am technically a noob on this space, I am approaching it as a first timer on server and instrumentation. This turned what felt like booking a colonoscopy into having chocolate delivered to your home.

Dec 4

Not a paid ad. I love Honeycomb.io - works so seamlessly with .NET.

124

"printf magically becoming superman" (goes off to the sticker factory) 😍🌈✨

Quote Tweet

Dec 5

Replying to @monkchips

James I just got started, so I have close to no experience, but honeycomb made me not dread the process, it was just like printf magically becoming Superman. twitter.com/migueldeicaza/…

Replying to

James I just got started, so I have close to no experience, but honeycomb made me not dread the process, it was just like printf magically becoming Superman.

Quote Tweet

Dec 4

Replying to @MartinDotNet @maggiepint and @_cartermp

It has paid off. That onboarding felt like it was second nature. No friction. I sent Phillip my notes for a couple of tiny tiny very tiny hiccups - because he asked for any feedback and I felt obligated :-)

oh, this is so cool 🙌

@BennettGarner

took my engineer/manager pendulum piece and unpacked each point, adding more details, advice and context overall. great article! medium.com/developer-purp

Quote Tweet

Bennett Garner

@BennettGarner

Dec 3

I love the idea of unconventional #developer career paths. S/o to @mipsytipsy for inspiring this article. Glad to see it getting love here on Twitter. twitter.com/alexsobrino/st…

Jeremy Daly

@jeremy_daly

Nov 30

Ran into this amazing person signing her Observability Engineering book!

Jessica Joy Kerr

@jessitron

Nov 30

“I used to say: test the happy path first, then fill in the rest gradually. Now I say the opposite. The happy path, you hit all the time. Test the paths that you don’t see every day.”

@tastapod

#yowlondon #testInProd

Show this thread

Jessica Joy Kerr

@jessitron

Nov 30

I like that “predictable” includes observability;

@tastapod

mentioned

and building-in the figuring-out. #yowlondon

Show this thread

Thor Hovden

@thor574

Nov 28

Good talk; getting continous deployment to 15 mins (++) will grow great teams. Make tests, code, get green lights, review. From when you merge to master, get it to prod within 15 min.

Quote Tweet

Nov 28

If you want to watch my talk on why it is (finally) time to fulfill the promise of CI/CD, go here -> youtu.be/XppRCB99djw?li (tagline: "deploys in 15 minutes or bust!", "software development death spiral")

P99CONF

@P99CONF

Nov 26

Observability and service-level objectives can help get you to "Game of the Year". 👀

@honeycombio

shares the performance lessons we can all learn from game developers during her #P99CONF session. Check out her talk on-demand: bit.ly/3DzrPsV #ScyllaDB

newsletter.attainable.engineer

Attainable

@attainableeng

Nov 26

A 🙏thank you for my profiles in software series. This has been a great pleasure and I look forward to many more!

Profiles in software: Better than I dreamed

A quick thank you to the group of folks that helped me with this little project

Replying to

é muito triste isso, eu vou sempre de encontro com a Charity:

Quote Tweet

Oct 14, 2019

treat every alarm like a heart attack. _fix_ the motherfucker. i do not care if this causes product development to screech to a halt. amortize it over a slightly longer period of time and it will more than pay for itself. twitter.com/brian_scanlan/…

Olly Headey

@lylo

Nov 23

“The times I have come closest to burnout or flaming out have never been when I was working the hardest, but when I cared the least. Or when I felt the least needed.📈📉💔” This is powerful writing from ⁦

The Hierarchy Is Bullshit (And Bad For Business)

⁩

charity.wtf

My friend Molly has had an impressive career. She got a job as a software engineer after graduating from college, and after kicking ass for a year or so she was offered a promotion to management, w…