ResearchOct 21, 20266 min read

Walk-forward validation, honestly.

A single in-sample / out-of-sample split is one draw from a noisy process. Rolling the window forward tells you far more.

Every strategy looks good on the data it was fitted to. The question that matters is how it behaves on data it has never seen — and how stable that behaviour is.

One split is one sample

A classic 70/30 split gives you exactly one out-of-sample period. If that period happened to be friendly, the result says more about the regime than about the idea.

Rolling the window

Walk-forward validation repeats the experiment: fit on a training window, evaluate on the following test window, then slide both forward and do it again. You end up with a distribution of out-of-sample results rather than a single number.

Choose a training length that matches how often you would realistically refit.
Keep a gap between train and test if features use overlapping horizons.
Charge realistic costs and slippage in every test window — not just at the end.

Reading the result

Look at the spread, not just the mean. A strategy whose Sharpe ratio is 1.5 in every window is a very different proposition from one that averages 1.5 by alternating between 4 and −1.