Monte Carlo and overfitting in EA backtests

Write down the simulation assumptions

Record source trades, cost convention, risk mode, random seed, path count and horizon. Percentiles describe this chosen model and data. For fixed monetary outcomes, changing the order cannot change the final sum. Bootstrap sampling can change it by repeating or omitting outcomes. Neither method creates a historically unseen loss mechanism automatically.

Look at dependence and sizing

Losses may cluster and positions may overlap. Independent resampling can erase that structure; block methods retain some dependence but introduce block-length choices. A simulation of closed trade outcomes is not floating equity. Recalculating current-balance sizing also requires consistent stop and volume information; merely stretching a money curve does not reproduce the EA.

Review the selection process too

A simulation around one selected winner does not account for every discarded preset. Retain the trial count and test a justified parameter neighborhood. A wide acceptable region is more useful than one sharp peak, but cannot certify absence of overfitting. Keep independent evaluation protected and disclose when a result has already influenced selection.

Checks to make before deciding

• Select a small justified neighborhood around the frozen preset. Name the parameters and ranges before viewing results; separate robustness checks from a new optimization campaign.
• Keep all other settings and data fixed. Change one dimension at a time initially, then inspect interactions where the strategy has a clear reason for coupled parameters.
• Show net return, equity drawdown and trade count across the grid. A smooth profit surface can still conceal unacceptable risk or too few observations.
• Include boundary values and small plausible perturbations. Inspect whether slight changes reverse the result, eliminate trades or trigger unexpected order behavior.
• Specify the source series, risk mode, method, random seed, path count and horizon. Retain simulation code and source hashes so the fan chart can be regenerated exactly.
• Distinguish shuffling without replacement from bootstrap sampling with replacement. Shuffling fixed monetary P/L preserves the final total while changing the order and drawdown path.
• A bootstrap can change final P/L by repeating and omitting outcomes. State the empirical distribution assumption and investigate whether rare losses are adequately represented.
• Check serial dependence and overlapping trades. Consider block resampling when clustered losses matter, and compare plausible block lengths rather than assuming independent trades.