Backtest vs forward test: a fair comparison

Freeze before inspecting

Save a dated executable and preset before examining the reserved period. Record whether that period influenced the strategy or the selection of candidates. Calling the latest year out-of-sample is not enough if it was already used to choose the winner. Once you tune to an observed evaluation result, that period is part of your research history.

Compare the actual overlap

Use the same executable, symbol, signal timeframe and risk basis. Compare the common verified dates, and keep starting capital, account currency and cash flows explicit. A long historical profit and a short forward profit are not comparable raw totals. Examine normalized outcomes, trade count, exposure and matched signals rather than relying on two attractive curves.

Keep discrepancies visible

Reconcile missed orders, slippage, different sessions and rejected requests. An unmatched trade can explain more than a headline profit factor. If both periods fail your predefined criteria, keep the evidence and classify the candidate accordingly. A favorable public chart should not be created by deleting unsuccessful tests or obscuring the account type.

Checks to make before deciding

• Declare development, selection and evaluation periods before examining the holdout. Store a dated manifest showing when the binary and preset were frozen.
• Keep the latest full year separate, but call it independent out-of-sample only when it demonstrably did not influence development or candidate selection.
• For walk-forward tests, specify training window, test window, step size and refit rule. Apply only information available at each refit and record all selected parameters.
• Distinguish historical forward, demo forward and real-money forward. Identify broker, account type, server, deposit changes, start date and evidence source without ambiguous live labels.
• Identity: B/F executable version and hash, preset hash, symbol and actual signal timeframe. Confirm equal trading rules before comparing money or percentages.
• Environment: B/F broker server, account currency, account model and leverage. Identify historical split, demo or real-money forward and any custom symbol settings.
• Timing: B/F requested dates, effective eligible dates, timezone and common overlap. Check missing coverage and warm-up before choosing the comparison interval.
• Sizing: B/F initial capital, reference capital, risk percent, sizing mode and position limits. State whether cash flows or manual lot changes occurred.