O.k. time for a mini-tweet thread which is all about network and TCP optimization! How can we make connections fast and reliable? and what's really going on anyway. It's all way too confusing so let's demystify it a bit.
-
-
That's not what's really going on. The internet and modern network pipes are actually queues, and that difference is important. Imagine instead that each pipe is really like a long train, going in a loop, and so long and full of railcars that the front and end meet.
Show this thread -
-
Maybe another metaphor that might be useful is each link or pipe is like it's own sushi conveyer belt.




Show this thread -
Now, let's think about the link with this model. We can data from one end to the other by putting it on one end and letting the belt/train get it there. We just have to wait for a free "slot".
Show this thread -
How long it takes to get there depends on two things: how soon a free slot comes up, and how far the distance is. But there's one more wrinkle: our packet might not make it all. It might fall off the train along the way! or it might have to wait too long for a slot and give up.
Show this thread -
O.k., so let's take this model and use to see how TCP works and why! So TCP, the transmission control protocol, takes data like our movie and moves it from A to B in order and reliably.
Show this thread -
It does this by breaking the movie into packets. Packets are better than sending the whole movie because if you're going to lose some data a long the way, it's better to resend only a small amount of it than the whole thing.
Show this thread -
Now we know our packets might not make it, so TCP says that packets have to be acknowledged, which just means that the other end says "yes I got your packet" when it gets a packet.
Show this thread -
So let's say we put a packet on the sushi belt, and it trundles along, and then it gets to the other end, and the other end takes it, and puts its own acknowledgment packet on the belt headed back to us. If we won't see that reply, we know to resend. O.k. great.
Show this thread -
That's actually enough to get some reliability, but it's really slow. We'd really like to fill the belt with packets, and get much more data across in the same amount of time. To do this, TCP has something called "windows".
Show this thread -
The window says "hey, feel free to send me this many packets, even if I haven't acknowledged them yet". Ideally you want the window to be as big as the number of free slots between you and the recipient.
Show this thread -
So TCP tries to find this value: it starts out slow, and increases until it detects a drop - a packet that didn't make it, which is assumes is because there aren't enough slots. When that happens it reduces (often by a lot) the size of the window. It sort of "homes in".
Show this thread -
On modern big fast networks, like AWS's, the default for this window size on common OS's is often too small, as is the number of "slots" in other layers. This is why increasing the TCP window size, and the TCP read/write buffers, and the ethernet queue lengths can be dramatic.
Show this thread -
O.k. the next thing our model shows is that acknowledging every packet is kind of dumb; sending back a sushi roll for every one that you receive is just wasteful. We can acknowledge what we got, and what we didn't get, in batches!
Show this thread -
The modern form of this is selective acknowledgements (SACK) ... where we can basically just scribble a note back on the sushi belt that tells the sender "hey, here's what I got and didn't get". The sender than re-transmit only what it needs to.
Show this thread -
So if you do those two things: make your window size big, and make sure selective acknowledgements are on, you can make a big difference to your performance! That video can get to you more quickly.
Show this thread -
Of course we do this kind of stuff ourselves for our own services, but if you're transferring data between your own machines or whatever, take a look!
Show this thread -
now let's extend the model: I said that a pipe or link is like a sushi belt, but there are lots of links interconnecting! So it's like a stadium full of sushi belts, with packets hopping belts. It's like Tim Burton and the Coen brothers made a movie together.
Show this thread -
when the belts interconnect, they might be moving at different speeds, or one might have less capacity than the other, so we have little holding areas, we call these "buffers".
Show this thread -
Generally packets enter and leave these buffers in order, but in some networks you can have priority lanes here, giving priority to some packets over others. At AWS, our belts move so quickly and there are so many free slots that we don't need to do this, it'd be pointless.
Show this thread -
But if there is congestion, and slots are busy, it's because senders are sending too much; it's key that they know quickly, so we make these buffers small. The problem of having these buffers too big is called buffer bloat (https://www.bufferbloat.net/projects/bloat/wiki/What_can_I_do_about_Bufferbloat/ …)
Show this thread -
This can all be very confusing without the right mental model. Because we generally want one kind of buffer - the window size - to be big, but another kind of buffer - the buffers between links - to be small.
Show this thread -
But if you see the network as train-cars or sushi on a belt, what you can see is that what we *really* want is to fill as many slots as we can when we're sending data! That's really all that's going on.
Show this thread -
One problem with the metaphor: packets don't actually go in loops, they come at the other end, so unlike a sushi belt, there's a kind of off-ramp at each end. Also packets only enter and exit at the ends. There's really no perfect metaphor.
Show this thread -
I'm going to meditate on better metaphors, so that's it for now :)
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.

