In my beautiful engineering world, you can roll out a deploy or feature flag a tiny amount into prod, see clear problems with metric cohorts, and halt or revert automatically. A girl can dream.
Conversation
How do you think about this with persistent data? The scariest upgrades I've ever done were ones that involved changes to data on disk. Sure, we could roll it out to a tiny bit of prod, but then those users were stuck until we got a fix out because there was no going back.
1
The right answer is clearly "always have a way to revert to the previous format" but... we don't always have the engineering capacity for that. 🙃