I swear that I’m going to grow old waiting for backtests to finish loading.
Do any of the pros have backtesting advice as far as efficient loading times goes?
@Ksidiii @goodalexander @KrisAbdelmessih @VolQuant
-
-
If you insist on Python — profile everything, find out where the bottlenecks are, and then use caching/vectorization/numpy/numba etc to seep up key sections. Also pandas is just intrinsically very slow, often better to work with values and cast back to dataframe at the end.
-
Out of curiosity
@macrocephalopod ; do you just mean stuff like apply (abs god forbid, manual looping) when talking about pandas performance being slow? I thought most of the “canned” transformations were supposed to be pretty performant - Show replies
New conversation -
-
-
What should I use? SQL?
-
Mostly a joke. Python is fine for most use cases, especially with vectorization and other speed ups. Would only consider using something else if you have many timestamps, many assets, or complex intermediate calculations that you can't cache.
End of conversation
New conversation -
-
-
What would you recommend instead? Thanks.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.