If I get time later (I wish you could buy time or at least rent it and die owing someone), I'll write up a quick blog on an interesting engineering challenge. Twitter produces on average 5-6k tweets per second. Decoding the tweets, extracting the hashtags and coming up with a
Conversation
real-time dashboard showing the top trending hashtags is an interesting challenge. This is basically another variation of the "Top-K" problem but doing it in Python fast enough to keep it real-time is a very interesting data engineering challenge.
Replying to
Less of a Top-K problem, more of a K-Pop problem. The top trending keywords going to be BTS
