O.k., this is happening ... now. I'm going to explain what's actually going on when data is encrypted, hopefully without mystifying, oh and some of the weird and inconsistent stuff cryptographers come up with.https://twitter.com/colmmacc/status/1100976962846154752 …
-
-
Instead of "scrambling" letters, modern encryption works by taking data and combining it with seemingly-random looking data in a clever way. It's similar to rot13 in two important ways: encryption and decryption are actually the same operation, and everything is in-place.
Show this thread -
Actually did you notice that rot13 is both an encryption and decryption algorithm? rot13(Ovaltine) -> Binygvar , rot13(Binygvar) -> Ovaltine. I like to think that's one of the nice symmetries of symmetric encryption. Anyway, back to that clever way I was talking about ...
Show this thread -
The cleverness we use is a bitwise XOR operation. There are inconsistent notations for XOR in cryptography and formal logic and code, but I'll use the one you're probably familiar with. It looks like a hat: ^
Show this thread -
XOR is short for "exclusive OR" and it's an operator (or function if you want to think of it like that) that takes two arguments and returns one result. a ^ b = c. It's bitwise, so it operates on each corresponding bit.
Show this thread -
If a and b are bytes, you can think of a ^ b = c as really 8 different operations all happening at once. ^ compares the first bit of a, and the first bit of b, and puts the result in the first bit of c. And it does the same 7 more times, for the other bits.
Show this thread -
The rules are simple: if a bit from a is "1" OR if a bit from b is "1" then we set the same bit to C to 1 ... but only if they *both* weren't 1. That's the exclusive part. Here's an old school truth table:
Show this thread -
-
Now the cool thing about xor is that it's like rot13. We can use to encrypt and decrypt things. Going to use some really simple examples here. Let's say that we want to encrypt just the number "3" and that our key is another number "7".
Show this thread -
So 3 ^ 7 = 4. So the encrypted version is "4". Now to decrypt, I just do the same again: 4 ^ 7 = 3. Try any numbers you like, or any data, this will always work: XOR always reverses itself.
Show this thread -
Byte by byte, that's how we're actually encrypting and decrypting things, there's no scrambling or moving going on ... but XOR-ing. The hard part is coming up with the data to xor with.
Show this thread -
One approach is to have a big chunk of secret data lying around and use that to XOR. As long everyone who needs to send or read the original plaintext has the same secret data ... this should work. Couple of problems with this:
Show this thread -
Problem 1: The secret data needs to be seemingly-random. You can't use the text from book or something. Any patterns in the secret data will show up in the encrypted version, and that's literally part how the allies beat the axis powers in WWII.
Show this thread -
Problem 2: You can't ever re-use the secret data. Again, patterns will show up. So you have to somehow securely get big wedges of secret random data (One time pads) around to everyone who needs them. Too hard.
Show this thread -
So in modern encryption, we *generate* the secret data we need from a small key, and those keys are much easier to get around and protect. This is what symmetric encryption algorithms really are: schemes for deterministically generating random data from a key.
Show this thread -
That "deterministic" part really matters: two people with the same key have to generate the *exact* same data, or else they won't be able to understand one another.
Show this thread -
You've probably heard of lots of these algorithms: AES, 3DES, DES, RC4, ChaCha20. All of these algorithms do this. It turns out that the math problem of taking a key and generating a random stream of data, one that has no patterns and is not predictable in any way, is hard.
Show this thread -
From that list, only AES and ChaCha20 are considered secure today, they others have been broken in interesting ways ... people figured out how to predict them. AES itself has a bit of patchy reputation, because ...
Show this thread -
Cryptographers: AES is the premier and most-analyzed encryption algorithm. Absolute gold standard!
Also Cryptographers: AES implementations in software (not hardware) are either insecure, or slow, or both. It wasn't designed with caching side-channels in mind.
Show this thread -
Don't worry too much if you didn't follow that. Just take this away: AES is mathematically awesome but hard to code, it's a good thing we almost always have hardware support to do it instead.
Show this thread -
... anyway ... moving on ... how do these encryption algorithms actually work? How do we take a key and securely generate a random stream of data? I'm going to simplify things a little here and talk about blocks.
Show this thread -
These algorithms take three inputs, and produce encrypted text. The inputs are the key, the plaintext ... and - surprise - something strange called an initialization vector. So for example: AES(key, IV, plaintext) -> encrypted_data
Show this thread -
The key and IV are combined to create a set of "starting conditions" for the algorithm; it's like an initial permutation, or scrambling, of some scrabble tiles. The same key and the IV will always produce these same starting conditions? Why do we have an IV, you ask? ...
Show this thread -
We have an IV so that we can encrypt multiple messages using the same key. Without an IV, each generated stream would always be the same, and that's no good. One of rules is that we can't re-use ever. So we need the IV to mix things up. But unlike the key, the IV can be public.
Show this thread -
So when you encrypt a message and send it to someone, you can send add "hey, here's the IV I used". Now it's still critical that we don't re-use the combination of key and IV, because those would generate the same random data.
Show this thread -
Two common ways to do this: 1) the IV is some kind of counter: we just increment it for every message. 2) the IV is generated randomly, and it's a big enough value that we don't worry too much about collisions. Anyway, I said I'd talk about blocks.
Show this thread -
So the key and IV are "mixed" or combined in some way to create starting conditions ... these conditions are really an initial "block" of random data. For AES128, the block is 128 bits, for AES256 it's ... 256, for ChaCha20 it's 512-bits.
Show this thread -
Now the magic and personality of the encryption algorithm really comes in. It's really all about how we generate one block after another, and each block is seemingly-random with predictable relationship to what came before or after, to someone who doesn't have the key.
Show this thread -
I'm not going to go too deep into how these work, but if you want to get a sense yourself: start by looking up linear congruential generators, which is a really simple function that can make a block of data "cycle" in a random non-repeating way.
Show this thread -
Then look up Feistel networks, which are sort of the next level of that. Look up S-Boxes then if you're more curious, and finally, take a look at how Salsa20 does the rotations that ChaCha20 uses. It's all more approachable than you might think!
Show this thread -
O.k. so now we know how random streams of data can be combined with plaintext to do encryption and decryption, and we sort of know how random streams of data are produced. Isn't that it?
Show this thread - 15 more replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.