It's ten years since we launched Amazon CloudFront! Building it was super fun, my first day at Amazon was spent getting stuck into the CloudFront routing software and making it actually work. A few weeks later, but before we launched, we hit my favorite bug ever ...
-
-
we built this gigantic live map of the internet ... for every network out there, we figure out what the correct ordering of CloudFront locations should be. E.g. if I'm Seattle, then it's something like SEA, PDX, SFO ... you get the idea ...
Show this thread -
The map is based on latency measurements and geo-data. For the geo-data, we had import a huge geodb (we have a few of these) and import them into the CloudFront routing later. That import happened in YAML format.
Show this thread -
Anyway, the bug was that the YAML parser ignored the negative signs ... so negative GPS coordinates became positive ones. We caught it before launch, obviously, but it's still my favorite bug because it was so subtle.
Show this thread -
The bug affects 75% of the Earth's surface, but by geographic coincidence or Greenwish winning the meridian wars, it just happened that our entire dev team was unaffected. Both Seattle and Dublin are in the "positive, positive" quadrant of the world!
Show this thread -
So we never experienced it ourselves, or our test cases, and we didn't notice it until we really started looking at other locations. I always think of that when I remember that even "dumb" and "obvious" bugs can be subtle and invisible because of your perspective.
Show this thread -
*Greenwich in my previous tweet. Anyway, fun bug! CloudFront had an incredible launch team, including 8 then-or-now Principal Engineers, which is staggering!
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.