I can't believe my dumbass has stuck with Spark for so long. Dask solves 99.9% of my use case.
-
-
-
Replying to @jlowin
I recently got to enjoyably play with
@OmniSci (h/t@randyzwitch) and have some use cases for next year. Now I gotta check out@PrefectIO, too (h/t@markov_gainz).2 replies 0 retweets 11 likes -
Replying to @generativist @jlowin and
Esp because I have this semi-packaged / partially reimplemented idea called vaquero for data cleaning that is *way* *way* more intuitive if it's just python native rather than PySpark based. Basically, this,https://github.com/jbn/modpipe
1 reply 0 retweets 1 like -
Replying to @generativist @jlowin and
Plus a smart error wrapper for interactive, iterative, and self-documenting data cleaning. My guess is that if your flows doesn't do it, it would fit well.
1 reply 0 retweets 1 like -
Replying to @generativist @jlowin and
Oh...
@markov_gainz, things suddenly make more sense,pic.twitter.com/6WUeoPQRSk
2 replies 0 retweets 3 likes -
Replying to @generativist @jlowin and
Haha yessss let me know if you want to pair up at some point in the new year - I always enjoy working through use cases with people
1 reply 0 retweets 4 likes
Absolutely! You know a bit about what's going on over by me, but once that get's resolved (hopefully soon), definitely want to start collaborating.
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.