How to test a trading system: correctness before confirmation
Use deterministic fixtures, causal replay, declared selection logs and reserved evaluation without claiming that unit tests prove an edge.
Keep two acceptance questions separate
Software tests ask whether the program implements its stated rules. Market evaluation asks whether those rules have adequate economic evidence under a declared design. A fixture can disprove a calculation immediately, but a thousand passing fixtures do not create a thousand independent strategy-validation samples.
Freeze the instrument identity, inputs, policy versions and accounting convention before starting. Save the exact code and source-data fingerprints with the result. If another reviewer cannot tell which settings produced a curve, a precise performance statistic cannot compensate for the missing experimental identity.
Use references without borrowing a guarantee
TimeSeriesSplit documents sequential training and evaluation partitions, including a configurable gap. That mechanism is not an automatic solution for overlapping trade labels or delayed feature availability. Choose the interval and exclusion rules from your actual information and outcome timing.
Bailey and co-authors' backtest-overfitting paper addresses selection across candidate strategies. It motivates recording the search, but citing it does not estimate your strategy's overfitting probability. An implementation of a particular estimator would need its own complete candidate inputs, assumptions and reproducible calculation.
A minimal correctness fixture suite
In a fictional event stream, send the same entry identifier twice, a cancellation after expiry, a partial fill and an exit acknowledgement arriving out of order. Declare the expected position quantity and cash after each event. The duplicate must not create a second intended position merely because it arrived through a different network attempt.
Add a bar whose high and low cross competing exit levels without ordering information. The expected result should reflect the frozen ambiguity policy, not whichever path wins. These constructed events exercise logic; they are not sampled market observations or a measured forecast of operational incident frequency.
Replay only information available at the time
Reconstruct features from their historical availability rather than today's revised dataset. Fit transformations on the permitted training interval and save their parameters. Keep instrument additions, removals, session clocks and missing observations in the replay contract. A survivor-only universe can change the interpretation even when calculations are otherwise correct.
Record the first permissible order time and declared fill model. Test sensitivity to plausible costs and execution constraints without selecting the most flattering assumption afterward. When a required historical field is unavailable, mark the relevant experiment untestable rather than pretending the present-day value was known.
Log development and preserve reserved evidence
Every material threshold, universe, entry or exit change belongs in the candidate log, including abandoned runs. Once a reserved interval has been inspected to revise the model, it is development information for that revision. Renaming it a holdout does not restore its untouched status.
Keep the selection objective separate from later diagnostic outcomes. Under an entry-focused design, a favourable exit-adjusted result must not silently rank entry candidates if the declared entry objective was different. A later policy experiment needs its own identity and evidence, not a replacement label on the original result.
Report the actual terminal state
A system can fail software correctness, complete validation with no acceptable model, remain untestable because data are missing or pass the declared research gates. These outcomes are not interchangeable. State which checks ran, which failed and which stage is still absent.
No production promotion follows from this worksheet alone. The completion package should include fixture outputs, source identities, candidate history, reserved-evaluation receipts and limitations. A verified negative result is more useful than an attractive curve produced by an unresolved accounting bug, and it should remain preserved for subsequent review.

Evidence handoff
| Check | Record | Boundary |
|---|---|---|
| Fixture test | Expected and observed deterministic state | Not an independent market sample |
| Development replay | Candidate log and causal inputs | Not untouched confirmation |
| Reserved evaluation | Frozen identity and recorded access | Not reusable after tuning |
Column definitions: test_id, fixture_or_market, code_fingerprint, data_fingerprint, policy_version, available_at, expected_state, observed_state, candidate_id, selection_objective, development_interval, reserved_interval, inspected_at, gate_result, remaining_stage.
No example trade or user record is stored in the download. Keep credentials and personal account identifiers out of shared copies.
Prefix and cash checks with a limited conclusion
The replay checks causal rolling values, strict crossover intentions, next-open fills and three-completed-bar exits against independent Decimal arithmetic. Four future suffix changes leave all earlier decisions unchanged. Bad IDs, times, OHLC, nonfinite values and row counts are rejected. The repeated run reproduces the same object, including its negative net cash. These are software/data tests over invented observations. They do not supply independent strategy-validation periods, unseen market states or proof of a profitable cause. The exact source, settings, runtime and data hashes make the test reproducible without strengthening its financial status.