Conversation

SBF
@SBF_FTX
7) One side-note here: what does it mean to 'upgrade' an exchange? _Sometimes_ it means you buy a bigger server. We are really fucking embarrassed if that happens to us. Why? Because if _that_ was the bottleneck, we should have done it a while ago.
1
11
SBF
@SBF_FTX
8) So for us, 'upgrade' almost always means 'parallelization'. The core problem of matching engines is: you can always use more computers to match, but that only works if you can process in parallel. Say Alice and Bob both submit orders to buy. a & b can be done in parallel.
2
10
SBF
@SBF_FTX
9) Risk checks are generally account specific, so you can just have a different computer handle each of theirs. But then you hit c & d: actually matching the orders. And the thing is, they're not independent. If Alice lifts an offer, Bob can't lift that same offer.
1
5
SBF
@SBF_FTX
10) At least, not if they're in the same market. So you can't have one computer handle Alice's matching and another handle Bob's, at least not trivially, or they'll clash with each other, both thinking they lifted the same offer. This is when it gets tricky to increase capacity
1
3
SBF
@SBF_FTX
11) e & f are somewhere in between. In some sense, f should be parallelizable: just have separate computers handle updates for each fill. But what if Bob simultaneously buys BTC/USD and BTC/USDT. Both increase his BTC balance.
1
3
SBF
@SBF_FTX
12) If they don't talk to each other, both will report too low of a BTC balance for him! So there's still some tricky work to do there on the interplay between similar but different fills. And if you can't parallelize it, then you're stuck going one at a time.
1
3
SBF
@SBF_FTX
13) And if you're going one by one and trading is really busy, the _single_ computer handling all fills notifications will get a bigger and bigger queue of things to notify. So at some point the real solution here has to be finding a way to do more than one at once.
1
7
SBF
@SBF_FTX
14) (FWIW, when
@Nishad__Singh
and Gary were talking about this today, the metaphor used was basically of a symphony with a conductor, and improvements were to the conductor that coordinated which computer processed which fills.)
2
14
SBF
@SBF_FTX
16) One missing thing from this: some of the 'clashing' when you try to parallelize things comes from updating the same single database too frequently; there's a data storage parallelization problem that mirrors the data processing one.
Twitter Web App
13
Likes
seslly
@seslly
Replying to
@SBF_FTX
and
@SBF_Alameda
Sounds like you guys got application level auto scaling and likely cluster level auto scaling to handle demand based loads. Sharding the DB sounds like it’s up next for increased DB throughput. Keep up the good work
1
This Tweet is from a suspended account. Learn more
SBF
@SBF_FTX
Replying to
@yigitt_sub
in fact I basically am not, unless you count shitty VBA code :P but I spend enough time project managing devs that I try to pick up on what's going on
1
2
Show replies
SBF
@SBF_FTX
Replying to
@_dwasse
the public market trades list doesn't have accounts associated with it (otherwise you could see other accounts' trading history publicly)
1
1
Show replies