macrocephalopod

The second point (sensitivity to assumptions) is important because you want to know if performance is extremely sensitive to e.g. latency, transaction cost, fill ratio assumptions that may be different in reality.

Third point is extremely important (and tragically often neglected) because without an accurate simulator, you may as well not bother running a backtest at all (and it will uncover bugs in either your simulation or live code)

Last point is important because you *really* want to know if your turnover/risk/exposure stats differ from your expectations. If you design a strategy based on a 1-2 day forecast and in simulation it holds positions for months, something is broken!

There are many ways to assess quality of an alpha/signal/strategy without running a full blown backtest. For example for binary signals you can look at event studies/markouts. For continuous signals you can regress future returns on your signal, or ...

... use factor regressions or quintile/decile charts. What these have in common is they measure what you care about, i.e. how good the signal is at predicting future returns.

Monetizing the prediction is the job of portfolio construction (where you trade off alpha vs. risk, costs and constraints).

The backtest is just to make sure that all these parts are working together as expected. In many ways a slow backtest can be a feature, because it discourages you from running hundreds of simulations and optimizing based on the results!

How would you study event based seasonality, without predominantly looking at backtest results? Say, you think that the market rallies before FOMC. It's straightforward to pick a 1d lookahead and run some statistical test, but that doesn't tell you when you should put it on.