My advice here was pretty standard - profile, find bottlenecks, use caching/vectorization/numpy/numba to speed up the hot code.
But even better advice is "do less backtesting" (as pointed out by @therobotjames)https://twitter.com/Overlevered_AM/status/1382201962687557638 …
-
Show this thread
-
Backtesting is not a research tool. It belong right at the end of the process and you use it for four things - 1. Final verification of your idea 2. Sensitivity to assumptions 3. Reconciliation vs live trading 4. Non-performance stats like turnover, max net/gross exposure etc
1 reply 8 retweets 41 likesShow this thread -
Only in the first of these do you care about performance, and only then to check that the simulation passes a "good enough" bar. Using the backtest to optimize parameters is the path to overfitting.
2 replies 1 retweet 18 likesShow this thread -
The second point (sensitivity to assumptions) is important because you want to know if performance is extremely sensitive to e.g. latency, transaction cost, fill ratio assumptions that may be different in reality.
1 reply 0 retweets 12 likesShow this thread -
Third point is extremely important (and tragically often neglected) because without an accurate simulator, you may as well not bother running a backtest at all (and it will uncover bugs in either your simulation or live code)
1 reply 0 retweets 10 likesShow this thread -
Last point is important because you *really* want to know if your turnover/risk/exposure stats differ from your expectations. If you design a strategy based on a 1-2 day forecast and in simulation it holds positions for months, something is broken!
1 reply 0 retweets 9 likesShow this thread -
There are many ways to assess quality of an alpha/signal/strategy without running a full blown backtest. For example for binary signals you can look at event studies/markouts. For continuous signals you can regress future returns on your signal, or ...
1 reply 0 retweets 14 likesShow this thread -
... use factor regressions or quintile/decile charts. What these have in common is they measure what you care about, i.e. how good the signal is at predicting future returns.
2 replies 0 retweets 15 likesShow this thread
Monetizing the prediction is the job of portfolio construction (where you trade off alpha vs. risk, costs and constraints).
-
-
The backtest is just to make sure that all these parts are working together as expected. In many ways a slow backtest can be a feature, because it discourages you from running hundreds of simulations and optimizing based on the results!
3 replies 2 retweets 30 likesShow this threadThanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.