What about using highly efficient single node stream processing framework rather than the current distributed frameworks ? https://lsds.doc.ic.ac.uk/blog/do-we-need-distributed-stream-processing …
-
Show this thread
-
Of course this limits the scale capacity, but the limit could be high enough to fit many use cases.
1 reply 0 retweets 0 likesShow this thread -
Replying to @nicolasmonchy
Fine for limited use cases, but forces users/adopters to roll their own distribution when it hits capacity... This was and remains the Achilles heel of non-distributed CEP engines...
1 reply 0 retweets 1 like -
Replying to @darachennis
Yes, but if we are talking about dozen of million events / s it's more than limited use cases IMO. I wouldn't be surprised if half of the Spark Streaming projects could get rid of the distribution and the complexity that comes with it.
2 replies 0 retweets 0 likes
True too, far too many frameworks sold on the back of capability to word count text! Single node embeddable streaming is extremely useful, but so is distributed. And, legacy CEP systems still have superior aggregation and temporal pattern matching support. YMMV!
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.