We quickly figured that we'd be patching everything that day, so an emergency was declared and all AWS software deployments were paused. This is incredibly disruptive, but the call leader has the authority to do this on their own. Our CEO and SVP agreed with the call.
-
-
Unfortunately we had a few customers stuck though; their OpenSSL libraries were embedded in commercial software that they couldn't quickly upgrade. One of our VPs reached out "Is there anything we can do here?"
Show this thread -
So at about 2AM, I wrote a Netfilter plugin that could block heart bleed using the Linux Kernel firewall. It's still on GitHub ... https://github.com/colmmacc/nf_conntrack_tls … , it tracks the TLS record layer state machine and would drop any heartbeat messages. Crude but effective.
Show this thread -
In our annual planning, we had raised the idea of writing our own TLS/SSL implementation because we thought we could better, but it was a nascent plan. Well that went from nascent to DO IT NOW. I started writing when became Amazon s2n.
Show this thread -
It took about 5 weekends, just me, and there's something very special about finally getting a bunch of code together and seeing it work in a browser. It took a little longer, and 3 intense security reviews, to get approval to Open Source it, but our CEO was very supportive.
Show this thread -
Now it's widely used across AWS. Blows my mind to think that S3 is using it!https://github.com/awslabs/s2n
Show this thread -
s2n is coded specifically in a way to try to avoid the problem heartbleed hit. Rather than parse memory into integers using pointers directly, all across the code, s2n uses a "stuffer" data structure that includes a cursor. Similar to BoringSSL's crypto_bytes, or DJB's stralloc.
Show this thread -
Oh BoringSSL! In the months after HeartBleed, the industry rallied to get OpenSSL more funding and support through the core infrastructure initiative. We still take part! And the BoringSSL and LibreSSL forks of OpenSSL happened. Great work from each!
Show this thread -
The next year, the amazing
@BenLaurie and@trevp__ started an annual High Assurance Cryptography workshop after@RealWorldCrypto, that has also born fruits and helped us produce tools that can analyze cryptography code and find even subtle problems.Show this thread -
I'm almost done, but before I finish, I kind of depressing twist on this whole thing: The Heart Beat extension never really made any sense to begin with. A 0-byte record could have been used as a keep-alive, and ordinary path MTU discovery works for UDP!
Show this thread -
All of this trouble for a feature that to this day I can't even think of a good use case for. This is one reason why "Don't do less well. Do less, well." resonates with me as a motto.
Show this thread -
That's my story for now, until I remember something I forgot. Thanks to everyone who moved mountains 5 years ago. I'm in JFK waiting to fly to Bucharest, so AMA!
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.