Even with dockerized PySpark workflows, Dask has substantially lowered the friction for doing quick "I wonder" analyses. It's a wonderful tool. https://docs.dask.org/en/latest/
It also made me aware of just how slow my main archival HDD is.
-
-
That's *sorta* how mine is. SSD for most things; one fat HDD that I use as intermediary caching for things I'm actually working on; 32TB NAS for everything else.
- 1 more reply
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.