If a and b are bytes, you can think of a ^ b = c as really 8 different operations all happening at once. ^ compares the first bit of a, and the first bit of b, and puts the result in the first bit of c. And it does the same 7 more times, for the other bits.
-
-
You've probably heard of lots of these algorithms: AES, 3DES, DES, RC4, ChaCha20. All of these algorithms do this. It turns out that the math problem of taking a key and generating a random stream of data, one that has no patterns and is not predictable in any way, is hard.
Show this thread -
From that list, only AES and ChaCha20 are considered secure today, they others have been broken in interesting ways ... people figured out how to predict them. AES itself has a bit of patchy reputation, because ...
Show this thread -
Cryptographers: AES is the premier and most-analyzed encryption algorithm. Absolute gold standard!
Also Cryptographers: AES implementations in software (not hardware) are either insecure, or slow, or both. It wasn't designed with caching side-channels in mind.
Show this thread -
Don't worry too much if you didn't follow that. Just take this away: AES is mathematically awesome but hard to code, it's a good thing we almost always have hardware support to do it instead.
Show this thread -
... anyway ... moving on ... how do these encryption algorithms actually work? How do we take a key and securely generate a random stream of data? I'm going to simplify things a little here and talk about blocks.
Show this thread -
These algorithms take three inputs, and produce encrypted text. The inputs are the key, the plaintext ... and - surprise - something strange called an initialization vector. So for example: AES(key, IV, plaintext) -> encrypted_data
Show this thread -
The key and IV are combined to create a set of "starting conditions" for the algorithm; it's like an initial permutation, or scrambling, of some scrabble tiles. The same key and the IV will always produce these same starting conditions? Why do we have an IV, you ask? ...
Show this thread -
We have an IV so that we can encrypt multiple messages using the same key. Without an IV, each generated stream would always be the same, and that's no good. One of rules is that we can't re-use ever. So we need the IV to mix things up. But unlike the key, the IV can be public.
Show this thread -
So when you encrypt a message and send it to someone, you can send add "hey, here's the IV I used". Now it's still critical that we don't re-use the combination of key and IV, because those would generate the same random data.
Show this thread -
Two common ways to do this: 1) the IV is some kind of counter: we just increment it for every message. 2) the IV is generated randomly, and it's a big enough value that we don't worry too much about collisions. Anyway, I said I'd talk about blocks.
Show this thread -
So the key and IV are "mixed" or combined in some way to create starting conditions ... these conditions are really an initial "block" of random data. For AES128, the block is 128 bits, for AES256 it's ... 256, for ChaCha20 it's 512-bits.
Show this thread -
Now the magic and personality of the encryption algorithm really comes in. It's really all about how we generate one block after another, and each block is seemingly-random with predictable relationship to what came before or after, to someone who doesn't have the key.
Show this thread -
I'm not going to go too deep into how these work, but if you want to get a sense yourself: start by looking up linear congruential generators, which is a really simple function that can make a block of data "cycle" in a random non-repeating way.
Show this thread -
Then look up Feistel networks, which are sort of the next level of that. Look up S-Boxes then if you're more curious, and finally, take a look at how Salsa20 does the rotations that ChaCha20 uses. It's all more approachable than you might think!
Show this thread -
O.k. so now we know how random streams of data can be combined with plaintext to do encryption and decryption, and we sort of know how random streams of data are produced. Isn't that it?
Show this thread -
For full disk encryption, it almost is really. We basically just encrypt every block or sector of storage, under the same key, and using an IV that's derived from the "position" on disk. So we can always just decrypt any block anywhere on the disk, as long as we have the key.
Show this thread -
But there's a problem with this ... someone can mess with the encrypted data. If I change the value of any byte, even if I don't have the key, it will mess with the decrypted output. There's no real security against that kind of meddling.
Show this thread -
For sending messages and data over the network, that's not going to got it. We don't want people tampering with our information. So we need to add an integrity check! There's a few schemes for this.
Show this thread -
HMAC, GCM and Poly1305 are the common ones in use right now. In each case, these algorithms basically take the data as input, along with another key (an integrity key) and produce a MAC or a tag, which is just another piece of data that acts as a signature.
Show this thread -
So to encrypt, and protect, our string, one scheme might be: AES(key, IV, "Ovaltine") -> encrypted_output HMAC(key, encrypted_output) -> MAC and then on the wire, we send: IV | encrypted_output | MAC
Show this thread -
To decrypt, we check the MAC first by generating it again and making sure they are the same, and then we decrypt the output. Internally there are differences between how HMAC, GCM and Poly1305 generate these signatures, but you don't need to worry about that.
Show this thread -
Today, this combination of operations is wrapped up in function we call "AEAD" which means Authenticated Encryption with Additional Data, and it does all of this is a mostly-foolproof way for you. Basically: AEAD(key, IV, plaintext, additional_data) -> IV_encrypted_data_MAC
Show this thread -
The "additional data" is just any other data you might want to "prove" the sender has, but not send; like say some meta-data that establishes a permission. It's often left empty.
Show this thread -
Now you can still screw up with AEAD. If you re-use the same IV, that's bad!! There are attempts to make this better, my colleague Shay has been working on a cool scheme called SIV, and it adds a measure of protection against that too.
Show this thread -
If you do use unique IVs, modern encryption is really robust. In general, you could publish some encrypted text in the New York Times, and no-one will be able to crack it. This Is true even if /some/ of the text is known. For example ...
Show this thread -
In internet protocols a lot of the text is known, a HTTP server always responds the same way and the first few bytes are known and totally guessable. This doesn't matter at all - doesn't help an attacker figure anything else out even one bit. We've come a long way from WWII.
Show this thread -
But there are attacks that do work! If you're sending this data over a network, and someone can see the timing and size of message. This opens us up to traffic analysis.pic.twitter.com/8qeI9A3Ozp
Show this thread -
Let's look at length first. O.k. so the length is obviously not hidden. That's fine if you're trying to protect your password or credit card number in the middle of a response. No big deal. But it does mean that someone might be able to fingerprint the content you're sending.
Show this thread -
Simple example: if you send a gif over a messaging app, if the size of that gif is unique, someone in the middle can probably guess what gif you just sent. There are more sophisticated versions of this for Google Maps, Netflix, WikiPedia, and so on.
Show this thread -
The way we protect against this is to "pad" messages, to make large numbers of messages appear to be the same size no matter what. Military grade network encryption actually pads all traffic all the time, so it's always the same!
Show this thread - 6 more replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.