What about using highly efficient single node stream processing framework rather than the current distributed frameworks ? https://lsds.doc.ic.ac.uk/blog/do-we-need-distributed-stream-processing …
-
-
Yes, but if we are talking about dozen of million events / s it's more than limited use cases IMO. I wouldn't be surprised if half of the Spark Streaming projects could get rid of the distribution and the complexity that comes with it.
-
However, the distribution is more needed for batch jobs, and these tools are making a good job to unify batch/streaming APIs (see Beam, no more distinction)
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.