of course it involves YAML, and may be the root of my hatred for that format! So CloudFront is a DNS based CDN, the "magic" happens at the DNS layer, where our DNS servers figure out what the best location to serve your traffic is ...
-
-
Show this thread
-
we built this gigantic live map of the internet ... for every network out there, we figure out what the correct ordering of CloudFront locations should be. E.g. if I'm Seattle, then it's something like SEA, PDX, SFO ... you get the idea ...
Show this thread -
The map is based on latency measurements and geo-data. For the geo-data, we had import a huge geodb (we have a few of these) and import them into the CloudFront routing later. That import happened in YAML format.
Show this thread -
Anyway, the bug was that the YAML parser ignored the negative signs ... so negative GPS coordinates became positive ones. We caught it before launch, obviously, but it's still my favorite bug because it was so subtle.
Show this thread -
The bug affects 75% of the Earth's surface, but by geographic coincidence or Greenwish winning the meridian wars, it just happened that our entire dev team was unaffected. Both Seattle and Dublin are in the "positive, positive" quadrant of the world!
Show this thread -
So we never experienced it ourselves, or our test cases, and we didn't notice it until we really started looking at other locations. I always think of that when I remember that even "dumb" and "obvious" bugs can be subtle and invisible because of your perspective.
Show this thread -
*Greenwich in my previous tweet. Anyway, fun bug! CloudFront had an incredible launch team, including 8 then-or-now Principal Engineers, which is staggering!
Show this thread
End of conversation
New conversation -
-
-
Machine-generated YAML should not be minimal. Sounds like a story for https://noyaml.com/ ,
@GeoffreyHuntley - 1 more reply
New conversation -
-
-
love these stories
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.