I can't believe my dumbass has stuck with Spark for so long. Dask solves 99.9% of my use case.
-
-
-
Replying to @jlowin
I recently got to enjoyably play with
@OmniSci (h/t@randyzwitch) and have some use cases for next year. Now I gotta check out@PrefectIO, too (h/t@markov_gainz).2 replies 0 retweets 11 likes -
Replying to @generativist @jlowin and
Esp because I have this semi-packaged / partially reimplemented idea called vaquero for data cleaning that is *way* *way* more intuitive if it's just python native rather than PySpark based. Basically, this,https://github.com/jbn/modpipe
1 reply 0 retweets 1 like -
Replying to @generativist @jlowin and
Plus a smart error wrapper for interactive, iterative, and self-documenting data cleaning. My guess is that if your flows doesn't do it, it would fit well.
1 reply 0 retweets 1 like -
Replying to @generativist @jlowin and
Oh...
@markov_gainz, things suddenly make more sense,pic.twitter.com/6WUeoPQRSk
2 replies 0 retweets 3 likes -
Replying to @generativist @OmniSci and
Awesome! We’d love to see it. Give us a shout if we can be helpful or you’d like access to any of the Cloud tooling
1 reply 0 retweets 2 likes
Thanks!
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.