Backtesting evidence

SSO · report.md

Derived research evidence · docs/backtesting/evidence/sso/report.md

SHA-256: fb557fa425b8e358504852f160d9d9152df3110c38873945ef1d32d6d2f2375c

Original source text. A superseded report retains its original assertions; it does not replace a corrected finding. Reproduction code is displayed without execution.

# SSO Opus 5.5 Swing 15m — Backtest Report

## Verdict: INDETERMINATE (and NOT a near-miss)

Validation has n = 3 completed trades, far below the frozen bar of n >= 30, so
the verdict is INDETERMINATE regardless of the mean. The validation mean is
also negative (-1.28% net per trade, win rate 0%), so this is not a near-miss:
two of the three frozen conditions fail outright (trade count, positive mean);
only the total-return leg passes, and for a non-edge reason explained below.

## What was tested

TrendSpider's "SSO Opus 5.5 Swing 15m" rule, exactly as frozen in spec.md:
SSO only, 15-minute regular-session bars, long only. Entry at the next bar's
open after a signal bar where SMA(50) > SMA(200) and ADX(10) > 20, plus either
Setup A (close above the prior 14-bar high with volume >= 200% of its 20-bar
average) or Setup B (session VWAP crossed upward with RSI(14) > 50 and volume
>= 150% of its 20-bar average). Exits: -3.5% stop, +15% take profit, 7%
trailing stop, exit after 25 candles if PnL < 0%, exit after 60 candles if
PnL < 1%, checked in that order. Costs: gross and net of 10 bps round-trip.

## Data window (the binding limitation)

- Yahoo Finance SSO 15m, request 2026-09-06 to 2026-11-04 (59 days; a 60-day
  request is rejected, and range=max on intraday silently returns monthly).
- Bars actually covered: 2026-09-08 through 2026-11-03, 1,066 bars.
- 41 complete sessions (26 regular bars each, 09:30–15:45 ET starts), zero
  gaps, zero missing bars.
- Split: in-sample = first 24 sessions (2026-09-08 to 2026-10-09); validation
  = remaining 17 sessions (2026-10-12 to 2026-11-03).
- HONESTY CONSTRAINT: the page's May '22–Nov '26 claim (153% strategy vs 111%
  buy-and-hold) CANNOT be reproduced from Yahoo 15m data under the 60-day
  cap. This verdict covers the recent-window replication ONLY and neither
  confirms nor refutes TrendSpider's full-period claim.

## Results

| | IS (24 sessions) | VAL (17 sessions) | ALL |
|---|---|---|---|
| Completed trades | 5 | 3 | 8 |
| Win rate (net) | 20.0% | 0.0% | 12.5% |
| Mean gross / trade | -0.55% | -1.18% | -0.79% |
| Mean net / trade | -0.65% | -1.28% | -0.89% |
| t-stat (net) | -1.79 | -2.54 | -3.00 |
| Cumulative net | -3.23% | -3.79% | -6.90% |
| Max drawdown (trade-end equity) | -3.77% | -3.79% | -7.42% |
| SSO buy-and-hold, same window | +2.14% | -6.58% | -4.49% |

Per-leg (completed trades; no signal satisfied both setups):

| Setup | ALL n / mean net / win | IS n / mean net | VAL n / mean net |
|---|---|---|---|
| A (breakout + volume) | 4 / -0.42% / 25% | 4 / -0.42% | 0 / — |
| B (VWAP cross + RSI + volume) | 4 / -1.36% / 0% | 1 / -1.59% | 3 / -1.28% |

Exit reasons (all 8 trades): 7 x time_25, 1 x time_60. No stop, take-profit,
or trailing exit ever fired: trades drifted down slowly (-0.5% to -2.3% net)
and were time-exited, never reaching -3.5% or +15%.

## Reading the validation numbers honestly

- The validation total (-3.79%) is "ahead" of validation buy-and-hold
  (-6.58%), but that is not edge: the strategy was invested for only 3 short
  trades across 17 sessions and sat out most of a falling window. Being less
  invested during a down window is exposure, not selection skill — and the
  per-trade mean is negative.
- Signal rarity is itself a finding: across 867 indicator-valid bars, only 14
  bars satisfied the full entry filter (6 Setup A, 8 Setup B), and 6 of those
  were suppressed because a position was already open, leaving 8 trades.
  A rule that fires ~once every 5 sessions cannot be validated on a 59-day
  window; it would need roughly a year of 15m data to reach n = 30 at this
  fire rate, which Yahoo's 15m cap cannot supply in one pull.
- In-sample agrees with validation (both negative), so there is no
  in-sample/validation contradiction to resolve — but with n = 5 and n = 3,
  neither leg is powered.

## Hand check: PASSED exactly

verify_hand.py independently recomputed the first completed trade
(signal 2026-09-18 14:45 ET, Setup A) from klines.json with separate loop
code: SMA50 69.0683 > SMA200 68.9993, ADX(10) 23.972 (> 20, matching the
engine to 6 decimals), close 69.3586 above prior 14-bar high 69.2300, volume
ratio 3.407 (>= 2.0). Entry fill 2026-09-18 15:00 ET open 69.3500; exit walk
first fires time_60 at the 2026-09-24 09:30 ET close (83 bars held,
PnL +0.61% < 1%); exit fill 2026-09-24 09:45 ET open 69.8100; gross +0.663%,
net +0.563%. Every field matches trades.csv exactly.

## Assumptions flagged (could change results, verdict-robust)

1. Volume SMA(20) includes the signal bar. Excluding it would shift volume
   ratios slightly; it cannot plausibly create 27 missing validation trades.
2. RANGE(14) high excludes the signal bar (prior 14 bars). Including the
   signal bar would make Setup A nearly unfirable (close above a high that
   includes its own high).
3. The 2026-10-13 Setup B signal fires at the 09:30 ET open bar, where the
   "prior bar" VWAP belongs to the previous session (VWAP resets daily).
   Under a same-session-only VWAP-cross reading that trade would not exist:
   validation n would be 2 instead of 3, mean net -1.42% — verdict unchanged.
4. Data anomaly NOT cleaned: a few bars carry volumes far above the median
   (~67k), e.g. 2026-11-03 14:45 ET with 13,053,077 shares. These affect
   volume ratios and session VWAP exactly as Yahoo delivered them.
5. Buy-and-hold basis: last-window close / first-window open - 1; strategy
   equity compounds completed-trade net returns with idle periods at 0.

## Verdict against the frozen bar

- Validation n >= 30: FAIL (n = 3).
- Validation mean net significantly positive: FAIL (-1.28%, t = -2.54).
- Validation total net ahead of buy-and-hold: PASS (-3.79% vs -6.58%),
  by low exposure, not by edge.
- **Verdict: INDETERMINATE.** No Terminal preset, vote, or build follows
  from this test. A powered retest would need a longer 15m history than
  Yahoo's 60-day cap provides (a paid intraday feed, or stitched
  multi-window pulls with continuity checks) — that is a data decision, not
  a conclusion about the strategy.

## Files

- spec.md — frozen rule and interpretations
- engine.py — backtest engine
- verify_hand.py — independent hand check (PASSED)
- klines.json — 1,066 bars with indicators
- trades.csv — 8 completed trades, 0 incomplete
- stats.json — full statistics
connecting…