Chart Replay, Backtesting and Journaling

First Officer · C06: Strategy Construction · About 30–35 minutes

Safety rule: A research idea is not proven profitable. Predefine the stop, costs and conditions that invalidate it.

Your mission today

By the end you can build a no-lookahead trade ledger and calculate net expectancy after costs.

Before you start

Complete C05 and your Second Officer cash-risk worksheet. Have a demo or synthetic replay. Every number below is an imaginary teaching calculation.

1. The idea in plain words

A backtest checks what a fixed rule would have done under stated historical data and execution assumptions. It does not establish future profit. A good test records every eligible observation, losing attempt, skipped event, fill assumption and cash result, then separately evaluates data not used to choose the rules.

A backtest asks a narrower question than “would I be rich?”

Backtesting asks what a fixed rule would have done on a specified historical dataset under a specified execution model. It cannot prove what will occur in future markets. A result is informative only if the data, methodology and decisions are reproducible. Start by recording the symbol, broker/source, timezone, period, missing candles, spread history if available, contract mechanics and any data transformations. Split the chronology before parameter tuning. For example, reserve the latest consecutive 25% as a holdout and use earlier data for development. The percentage is an illustrative protocol choice; the essential idea is that future data cannot influence earlier rules.

A chart-replay exercise is conducted from the left edge forward. At each closed candle, the student records all indicators and structures computable then. If a swing is confirmed only two bars after its candidate point, it cannot authorize an entry at the candidate bar.

Loading the C06.5 teaching illustration…

Visual C06.5.1 — One candle at a time. A five-bar swing example can have its candidate high at bar 3 while confirmation requires bars 4 and 5 to finish. The pivot's price location belongs to bar 3, but its confirmation timestamp is bar 5. Testing an entry at bar 3 based on that confirmation is using information from the future.

In plain English: In a replay, don't ask “What would I do, knowing how the chart eventually ended?” Ask “What did I know after this candle closed?”

The same applies to full-day highs before the day ends, indicators recalculated using the current unfinished candle, or a breakout boundary determined using the future breakout bar. Reproducing the visible appearance of an entry on a retrospective chart is not enough; the rule must produce the same decision in chronological replay.

2. Worked example: loss before gain

In an imaginary set of 50 trades, 30 wins earn +1.2R and 20 losses lose −1R. Gross result: 30×1.2−20 = +16R. If cost is 0.15R for every trade, total cost is 50×0.15 = 7.5R, leaving +8.5R net. But on a separate 20-trade sample with nine +1.2R wins and eleven −1R losses, net after 3R costs is −3.2R. The second sample cautions against claiming reliable profitability.

Original Black Box Journey diagram; not a verified live strategy.

The ledger is the black box

Every offered event, not just every completed trade, needs a row. Minimum columns are: event ID; broker-time and normalized UTC time; symbol; timeframe; structural bias; setup state; confirmed trigger time; entry eligibility; planned Ask/Bid; stop and R at risk; target; expected spread, commission and slippage; fill status; actual exit/assumed resolution; highest unrealized favourable/adverse excursion; actual net R; and reason for rejection. Log missing history and ambiguous stop/target candles as distinct statuses. The row count itself is an important audit: a filtered strategy producing 20 trades in three years cannot be evaluated as if it has the same evidence as a strategy producing 2,000 independent opportunities.

The statistics that matter go beyond win rate. Net expectancy is average R after all modeled costs. Profit factor compares gross winning profits with absolute gross losses, but both can become unstable with small samples. Maximum drawdown shows the largest cumulative deterioration from a prior peak under a specified position-sizing assumption. Time under water measures how long the system spends below a prior equity high. Performance also needs a breakdown by year, session, direction, instrument and market regime. A pooled average may conceal that almost all gains came from one unusually favourable period.

Synthetic chronology of trading outcomesThe archival image is not included in this student edition.View the released teaching illustrations.{width=6.55in}

Figure C06.5A. This deliberately invented cumulative-R curve illustrates chronological separation. It is not a test of the Flight Path system.

3. Make the rule reproducible

Research log fields: versioned rules; eligible event timestamp; available completed bars; accepted or rejected gate; executed prices or assumptions; stop/target sequence uncertainty; gross and net R; costs; max adverse excursion; data period; and whether the sample helped design the rule. Preserve untouched later data for a true out-of-sample check.

Work the expectancy calculation

Imagine 50 synthetic completed trades: 30 winning trades at +1.2R each and 20 losing trades at -1R each. Gross outcome is 30×1.2 − 20×1 = +16R, or +0.32R per trade. If modeled round-trip transaction costs and slippage average 0.15R for every trade, total cost is 7.5R. Net result becomes +8.5R, or +0.17R per trade. This does not mean a 60% win rate is sufficient for any particular system: the payoff distribution, costs and likelihood of repeating the result matter.

Now reserve another set of 20 chronologically later synthetic trades, with nine wins at +1.2R and eleven losses at -1R. Gross = 10.8−11 = -0.2R. After 20×0.15R = 3R of costs, net = -3.2R, or -0.16R per trade. The favourable development sample does not override the negative holdout.

Loading the C06.5 teaching illustration…

Visual C06.5.2 — Put numbers side by side. The fictional development group contains 50 trades and ends +8.5R after the stated costs. The 20 later fictional holdout trades end -3.2R after costs. A profitable development period does not prove the strategy survives new data.

Try it yourself: Calculate 30 × 1.2 − 20 × 1 − 50 × 0.15 = +8.5R, and then 9 × 1.2 − 11 × 1 − 20 × 0.15 = −3.2R. These two results are different even though the rule is not changed.

Loading the C06.5 teaching illustration…

Visual C06.5.3 — See the sequence. The 70-trade net-R line comes from the exact same fictional ledger as the worked arithmetic, including 0.15R costs on every trade. It reaches +8.5R at trade 50 and finishes +5.3R after the negative holdout segment. That ending positive number does not rescue failed independent evaluation.

The correct conclusion is that the specific hypothesis has not shown robust evidence on the holdout. Do not use the failed holdout to alter parameters and then continue calling that same data “untouched evaluation.” You may define a new version and require a genuinely later unseen sample.

From proposed rule to falsifiable evaluationThe archival image is not included in this student edition.View the released teaching illustrations.{width=6.55in}

Figure C06.5B. Train, freeze, then evaluate. The mathematical example shows how costs change interpretation.

4. Count failures and skipped opportunities

Cost and path integrity

At small targets, the spread may consume a large share of the proposed edge. A long market entry ordinarily pays Ask and closes by selling at Bid; a short ordinarily enters by selling at Bid and closes by buying at Ask. Any model that marks a long's profit from a mid-price entry to a mid-price exit is optimistic unless it explicitly applies an execution-cost correction. Commission, overnight financing, conversion, rejected orders, minimum lot constraints and slippage also vary. A useful classroom stress test compares a baseline with wider spreads and adverse fill assumptions and asks whether the direction of net expectancy survives.

Intrabar path ambiguity is especially dangerous.

Visual C06.5.4: lookahead, stop-target path ambiguity and cherry-picked test resultsThe archival image is not included in this student edition.View the released teaching illustrations.

Visual C06.5.4 — Three easy ways to fool yourself. A future swing confirmation cannot be used early; a candle touching stop and target cannot prove the order without finer path information; and choosing the best of many trials without reporting all of them biases the result. The shown numbers are teaching examples.

If a historical M15 candle's high reaches the target and its low reaches the stop, the four OHLC fields alone cannot establish which happened first. Some backtest engines supply a specified execution model or finer data; students must understand its assumptions. MetaTrader 5 provides different tester modes and a separate forward-test period. A faster approximate mode should not be mistaken for proof of live executable fills. Our Academy research curriculum uses M15 as the lowest currency-entry signal timeframe; execution modeling may still need bid/ask information finer than that signal clock. Such data are for checking fills, not inventing an M1 signal strategy.

A complying loss remains in the study ledger. Never add a rule solely because one past losing trade was inconvenient.

5. Practise with the future hidden

Avoid the strategy-research trap

Testing dozens of slightly different pullback definitions on the same years almost guarantees that some appear excellent by chance. Keep a registry of every tested hypothesis, including abandoned failures. Do not silently pool correlated currency-pair trades as independent evidence; common USD or risk sentiment shocks can cause many positions to rise or fall together. Report simultaneous exposure and drawdown under portfolio rules, not just per-chart success. When the edge is small, a fragile profit factor, a negative year, or an out-of-sample collapse should lead to a cautious decision rather than a confident sales promise.

For the First Officer capstone, students must submit at least 30 sequential chart decisions and identify every no-trade opportunity; that number is an educational assignment minimum, not a statistical proof threshold. They must freeze a single version, report all offered signals, calculate net R with costs, and perform at least one chronological holdout comparison where available. Completion is judged by auditability, not whether the result is profitable. A negative or inconclusive outcome, properly documented, can earn full marks.

Independent research assignment

Write a two-page research note answering: What observation would make me reject this strategy hypothesis? Attach the frozen playbook, a chronological event ledger, a comparison with a simple baseline, a results table for development versus holdout, and a paragraph on why the conclusion might fail in live execution. Show year/session/long-short splits and worst drawdown. Report uncertainty; do not hide losing segments. When the available sample is too small or highly correlated, mark the conclusion insufficient evidence rather than stretching a confidence claim.

Glossary: lookahead bias = using information that did not yet exist at the decision time; out-of-sample = data reserved from model selection; expectancy = average profit or loss per trade after relevant costs; R multiple = outcome expressed as a fraction of the preplanned loss; overfitting = tailoring rules to historical noise; path ambiguity = unknown ordering of intrabar price events.

Common mistake

Rewriting an entry, stop or data window after seeing the outcome destroys the original test. Keep the first version and start a new, separately evaluated variant instead.

Plain-English glossary

• Backtest: Simulation of a frozen rule on prior data.
• Lookahead: Accidentally using future information.
• R multiple: P/L divided by predeclared initial risk.
• Expectancy: Average result per qualifying trade.
• Holdout: Data not used to design the strategy.

Try it yourself

Create a strategy card containing market, timeframes, entry, stop, exit and skip rules. Carry out the original independent exercise above in closed-bar replay. Record at least one rules-compliant loss and one rejected candidate.

What you can now do

• Build a no-lookahead trade ledger and calculate net expectancy after costs.
• State how the idea could fail and what it would cost.
• Keep a reproducible, non-cherry-picked evidence log.

Next lesson

CP01.1 · One Market. One Strategy. One Operating Plan..