macrocephalopod

Backtesting is not a research tool. It belong right at the end of the process and you use it for four things - 1. Final verification of your idea 2. Sensitivity to assumptions 3. Reconciliation vs live trading 4. Non-performance stats like turnover, max net/gross exposure etc

Only in the first of these do you care about performance, and only then to check that the simulation passes a "good enough" bar. Using the backtest to optimize parameters is the path to overfitting.

The second point (sensitivity to assumptions) is important because you want to know if performance is extremely sensitive to e.g. latency, transaction cost, fill ratio assumptions that may be different in reality.

Third point is extremely important (and tragically often neglected) because without an accurate simulator, you may as well not bother running a backtest at all (and it will uncover bugs in either your simulation or live code)

Last point is important because you *really* want to know if your turnover/risk/exposure stats differ from your expectations. If you design a strategy based on a 1-2 day forecast and in simulation it holds positions for months, something is broken!

There are many ways to assess quality of an alpha/signal/strategy without running a full blown backtest. For example for binary signals you can look at event studies/markouts. For continuous signals you can regress future returns on your signal, or ...

... use factor regressions or quintile/decile charts. What these have in common is they measure what you care about, i.e. how good the signal is at predicting future returns.

Monetizing the prediction is the job of portfolio construction (where you trade off alpha vs. risk, costs and constraints).