Research note
Validation framework
Six interlocking analytics that check whether a signal predicts anything: composite IC, per-lens IC, polarity, quintiles, kill-switch, walk-forward.
1 min read
Six interlocking analytics, all run against the cached backtest. No look-ahead — each backtest point uses only data available on that day; forward returns are pulled after the fact for the analytics layer only.
- Composite IC. Does the headline score predict forward returns? At 5/20/60d.
- Per-lens IC. Which lens does the work? Which is noise? Sorted by 20d magnitude.
- Per-indicator IC + polarity. For each indicator, does its
Directionmatch what the data says? Outputs a keep / flip / drop recommendation per indicator. - Quintile bucketing. Sort backtest points into 5 buckets by composite score. Compute mean forward return per bucket. If the model works, top quintile beats bottom quintile monotonically.
- Plumbing kill-switch validation. Forward returns when plumbing tripped vs clear, with the edge in bps and hit rate for each.
- Walk-forward + out-of-sample. IC computed year-by-year, with a regression-slope trend (stable / decaying / improving). Plus a single 60/40 in-sample / out-of-sample split with a sign-flip-aware verdict.
The whole thing lives at /validation and runs in ~30-90 s on a 10y window once the cache is warm.
This article is also a chapter of the Education page.