Validation: Walk-Forward & Multiverse

When the exploration finishes and you see an algorithm with a Sharpe of 3, a 400% return and a ridiculously small drawdown, your natural reaction is excitement. Resist that impulse. Before trading any algorithm with real money, you need to understand the most silent threat in algorithmic trading: overfitting.

Walk-Forward (temporal validation)

The idea is simple but powerful: instead of training and evaluating the algorithm on the same full history, you split time into windows. In each window, the algorithm is «selected» or calibrated on one segment (in-sample) and evaluated on the next segment it has never seen (out-of-sample, OOS). Then the window advances and the process repeats. In the end, only the cumulative performance on the OOS segments matters, because that simulates what would happen in the real future.

[--- train ---][test]
       [--- train ---][test]
              [--- train ---][test]
                     [--- train ---][test]

   → se evalúa SOLO en los tramos "test" (datos nunca vistos)

Configuration parameters

Metrics returned by Walk-Forward

Multiverse (synthetic validation)

Walk-Forward tells you whether the algorithm held up in the single past that actually occurred. The Multiverse answers a different question: what if the market had taken a different path? Instead of splitting the real history, it generates N statistically coherent «parallel» markets: other paths the price could have traveled with the same volatility, trends and correlations. If your algorithm performs well across hundreds of those alternative universes, it's because it captured something real — not because it exploited an anomaly unique to the specific past you happened to have.

Configuration parameters

The main result from the Multiverse is a Score from 0 to 100. What each range means:

• Score ≥ 75 — Robust. The algorithm performed consistently across the vast majority of alternative universes. This is the minimum bar to consider trading it live.
• Score 50–74 — Moderate. Works well in some scenarios but not all. It may be valid, but needs more context: what scenarios does it fail in? Are they likely in the near future?
• Score < 50 — Fragile. It only worked in a small fraction of the simulated universes. It most likely captured an anomaly in the specific history. Do not trade it with real money.

DSR and Monte Carlo (related)

Walk-Forward and Multiverse do not act alone. They are part of a four-layer system. The other two are DSR and Monte Carlo, which you will also find in the platform. DSR (Deflated Sharpe Ratio) is applied automatically during exploration: it corrects the Sharpe of each candidate by the number of combinations tested, because if you test 10,000 combos, the expected Sharpe of the best one by pure luck is very high. DSR lowers that threshold statistically and discards those that don't justify their Sharpe given the search size. Monte Carlo analysis — which you can run later on a specific algorithm — does something different: it randomly reorders the actual trades and simulates thousands of possible sequences to stress-test the equity curve and probable maximum drawdown.

The robustness loop

These four layers are not redundant: they are orthogonal. Each one attacks a different axis of overfitting. Together they form the process TradingNote calls the «robustness loop»:

1. 1

   Genetic exploration

   You launch the exploration and the engine tests thousands of combinations of indicators and parameters, keeping the candidates with the best performance on the historical data.

2. 2

   DSR discards the lucky ones

   The Deflated Sharpe Ratio adjusts the statistical threshold by the search size and eliminates algorithms whose Sharpe does not exceed what you would expect by chance when testing so many combinations.

3. 3

   Walk-Forward verifies the temporal dimension

   Confirms that the algorithm held up on segments of the history it did not use for optimization. If performance collapses out-of-sample, the algorithm memorized, not learned.

4. 4

   Multiverse verifies the scenario dimension

   Confirms that the algorithm did not depend on the specific path the market took. If it works across hundreds of alternative synthetic markets, it is robust to the randomness of history.

What survives all four layers deserves your attention. It won't be the most spectacular of the bunch, but it will be honest.

Once your best candidates have passed the robustness loop, the next step is to explore the results in depth and visualize the exploration in the Neural Explorer. Continue in «Results and Neural Explorer».