I swear that I’m going to grow old waiting for backtests to finish loading.
Do any of the pros have backtesting advice as far as efficient loading times goes?
@Ksidiii @goodalexander @KrisAbdelmessih @VolQuant
-
Don’t use Python.
If you insist on Python — profile everything, find out where the bottlenecks are, and then use caching/vectorization/numpy/numba etc to seep up key sections. Also pandas is just intrinsically very slow, often better to work with values and cast back to dataframe at the end.
-
-
Out of curiosity
@macrocephalopod ; do you just mean stuff like apply (abs god forbid, manual looping) when talking about pandas performance being slow? I thought most of the “canned” transformations were supposed to be pretty performant -
yes, you're right. pandas is pretty performant. people who complain about pandas performance just don't know how to use it. they typically loop over dataframes with df.apply() or df.iterrows() and wonder why it is slow ...
- Show replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.