What is statistical significance?

What it means

In trading the question is whether a strategy has an edge or merely had a good run. With enough strategies tested against enough data, some will look excellent purely by chance — this is not a hypothetical, it is arithmetic, and it is why testing many ideas requires more evidence per idea, not less.

Why it matters

The standard defence is to hold data back. Fit and refine on one period, then test once on a period never used during development. A result that survives that is not proof, but it is dramatically better evidence than a result that has never left the data it was born in.

What this changes in practice

We ran exactly this check. The shipping configuration was developed on January to July 2026 and then tested on August to December 2025, which it had never seen. It returned +$154 and was positive in three months of five — profitable, but not the "green every month" the in-sample period suggested. That gap is what out-of-sample testing exists to reveal.

What it takes to tell a real edge from luck in a trading record is worked out in how many trades before a track record means anything?