2/3 Then how you make it resilient, super-scalable, micro-second fast, ... it's all about art.
I spent quite sometime in developing @bitfinex 's one, via many iterations and experiments.
Now it's a beautiful beast, capable of incredible performance and powering several products!
-
Show this thread
-
3/3 Many exchanges rely on traditional databases to handle matching. This is why during peak market activity they won't perform well and throughput is heavily limitated. Of course it requires to develop custom fault-tolerance, resilience, ... but it's simply the only correct way.
2 replies 0 retweets 24 likesShow this thread -
Replying to @paoloardoino
Cool. But only if I can write a response on why all current matching and reporting breaks or fails to meet expectations in the most commonly used tech stacks in crypto under load.
1 reply 0 retweets 1 like -
-
Replying to @paoloardoino @IamNomad
Very surprised to read 'exchanges using traditional databases for their matching engines'.
1 reply 0 retweets 2 likes -
Replying to @cyrii_MM @paoloardoino
Yeah it's what happens when you get a bunch of web 2.0 guys who think they can solve a problem with "the cloud man" and "nodejs"
1 reply 1 retweet 2 likes -
Replying to @IamNomad @paoloardoino
I would have thought, to deal with the throughput, especially sub second updates, something like in memory data grids, or kafka streams would be the first thing they think of
1 reply 0 retweets 1 like -
Replying to @cyrii_MM @paoloardoino
Kafka pipelines arenr fast enough for matching or reporting. neither is whatever Apache is calling their grid tech now. Real exchanges (think nyse) tend to use in memory caches for all objects. The rest is secret sauce that's too deep a topic for twitter
2 replies 0 retweets 2 likes -
This Tweet is unavailable.
-
This Tweet is unavailable.
All valid points. Queue is the tricky part and it's also the real secret sauce. Often reliable queues are the real bottlenecks. Using reliable multicast you can have separate listeners that write a log as fast as they can on a RAID10. Still experimenting on this myself.
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.