Rare evening technical thread: about distributed systems, exponential backoff, and the tragedy of the commons problem.
-
-
If everyone else is well behaved, that greedy client will probably get service more quickly, they'll jump the queue basically. But the overall recovery will be slowed down a bit by the aggressive tactics. Then, if this behavior spreads, it's disaster.
Show this thread -
Everyone retrying aggressively makes the whole situation worse, for everyone. It's a "tragedy of the commons" - everyone acting in their own narrow self-interest actually makes everyone worse off in the end.
Show this thread -
I often think of airport baggage belts, where people compete and huddles to get closer to the belt; we'd all be better off if we *all* just stood back a few meters. Clearer views, easier access, easier exit.
Show this thread -
Many distributed systems really rely on this kind of community oriented cooperation. TCP, BGP, the Web, e-commerce, Cloud Services. All have examples where a kind of codified socialism, fairness and restricting individual competition, keep the systems safe and cost-effective.
Show this thread -
It needn't be like that; industries don't have to co-operate! Stock markets are the opposite, where there's a costly (maybe wasteful) race to be the fastest and try very aggressively, producing HFT.
Show this thread -
Blockchains are another example, where there is deliberate aggressive competition, and it's even exploited to drive a perverse consensus based on mutual mistrust.
Show this thread -
Anyway, back to the tragedy of commons! Afaik, no-one has really found a robust solution to this beyond blocking mis-behaving clients before they spread, which also comes at expense.
Show this thread -
IME, how it "really" works is that the folks who builds clients and services have a sort of code of honor system. Everyone recognizes that it would be a wasteful race to the bottom, and mostly self-policing works. So browsers, SDKs, etc ... all do sane, safe, decent things.
Show this thread -
I don't really have a big technical lesson there, I just find it fascinating that huge sections of the economy can get by like that. It's very inspiring and reassuring! /EOF
Show this thread
End of conversation
New conversation -
-
-
Can’t we design the system to encourage good behavior and discourage greedy/selfish behavior? For e.g., by having a concept of “access cost” (which is based on various factors such as system load, state, etc.) that clients must “pay”?
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.