Invalidated on July 30, 2026. This replay used the pre-correction repaired price panel. It remains an engineering record, not performance evidence. See the corrected OOS audit.
Archive status: legacy one-session diagnostic; closed as the primary live experiment on 2026-07-28.
This experiment is retained for reproducibility and for the engineering lessons
it produced. It must not be mixed with the redesigned 20-session live evidence.
The consolidated decision and archive rationale are recorded in
docs/experiments/2026-07-28-live-horizon-decision.md.
Does the strategy displayed by the Live Forward Monitor show historical evidence when its own economic contract is replayed, instead of comparing it with the different weekly reference strategy?
This experiment intentionally reproduces the current monitor, including choices that are not recommended for production:
- one decision after every completed close;
- one-session next-open-to-open sector-neutral rank as the training label;
- previous-close-to-next-close return as the displayed evaluation;
- current decision-date members reused throughout that decision's training history;
- the broad available member universe, without the reference system's top-250 liquidity screen;
- Ridge and XGBoost tuned separately on five purged 42-session temporal folds;
- fresh sector-capped top-40 selection every day;
- 100% investment and equal weight;
- missing evaluated holdings reweighted across usable names;
- every sentiment feature set to zero;
- gross monitor-equivalent returns plus a separate 8-basis-point and $10-per-order cost view;
- SPY measured over the identical decision-close-to-next-close interval.
Labels are restricted to feature dates whose entire forward-open outcome was known after each historical decision close. The replay rebuilds the sector-neutral target inside the membership set used by that historical decision.
- Repaired adjusted OHLCV panel through 2023-12-29.
- Effective-dated inferred sector history.
- Effective-dated S&P 500 membership intervals.
- Separately cached adjusted SPY history.
- Predictor construction shared with
run_live_forward_snapshot.
The cached replay panel is ignored by Git and contains the live technical and cross-sectional feature schema, one-session training returns, next-close evaluation returns, and sectors.
python -m src.cli run-live-settings-backtest `
--model-family ridge `
--manifest configs/experiments/live_settings_backtest.yaml
python -m src.cli run-live-settings-backtest `
--model-family xgb_ranker `
--manifest configs/experiments/live_settings_backtest.yamlDate, output, and Optuna-trial overrides are available for compute-bounded diagnostics.
This experiment answers whether the current daily monitor contract had historical signal. It does not validate the weekly five-session reference strategy. Conversely, failure here does not invalidate the weekly strategy because horizon, execution, exposure, turnover, feature schema, and model-selection cadence differ.
The primary report is net of research costs. Gross monitor-equivalent performance is retained only to reconcile the dashboard. A positive point estimate is insufficient: active-return uncertainty, turnover, cost drag, parameter stability, score coverage, and skipped constant-score decisions must all be reported.
The first completed comparison uses 2023-11-20 through 2023-12-20 and five deterministic Optuna trials per model per decision. Five trials are below the normal live budget of 50, but are the minimum tested budget that consistently reaches a nonconstant XGBoost candidate. This run is a pipeline and directional diagnostic, not a final confirmation.
Ridge completed all 22 decisions without a model-selection failure.
| Metric | Ridge | SPY |
|---|---|---|
| Gross cumulative return | 4.69% | 4.44% |
| Net cumulative return | 1.19% | 4.44% |
| Gross annualized active return | +2.98% | — |
| Net annualized active return | -36.04% | — |
| Net Sharpe | 1.22 | 5.49 |
| Maximum drawdown | -2.64% | -1.40% |
The gross monitor-equivalent portfolio exceeded SPY by only 0.25 percentage point over the sample. Average daily one-way turnover was 116.29%, average orders were 62.27 per day, and the annualized modeled variable and fixed costs were 23.44% and 15.58%, respectively. The strategy therefore did not preserve its small gross edge after the project's existing execution model.
The validation process looked more favorable than realized trading: selected trials had positive top-40 validation excess on all 22 dates, with mean validation rank IC 0.00229. Daily portfolios still replaced a median 57.5% of names, and tuned Ridge alpha ranged from 4,033 to 11,495.
A 100-path sector-capped random-selection placebo produced median annualized net active return of -17.14%. Ridge's -36.04% was worse, with a one-sided p-value of 0.921. This diagnostic supplies no evidence that the current daily Ridge monitor monetizes winner selection after costs.
The five-trial XGBoost diagnostic was stopped after six attempted decisions. Four of six dates failed because no trial produced nonconstant scores in every temporal validation fold. Failed dates correctly carried the last valid portfolio after the replay's fail-closed correction.
This is an operational result about the reduced trial budget, not an XGBoost return result. Five trials are insufficient for this search space and broad universe. A performance comparison requires the configured 50 trials per date, checkpointed model artifacts, and enough compute to finish the full sample. No partial XGBoost return estimate is reported.
No additional performance run is required for this one-session contract. The project learned that daily forecast issuance can be retained without using a one-session target: a new 20-session forecast can be issued every day and one matching outcome will mature every day after the initial warm-up.
The archived design also exposed material contract problems: the training and
displayed return intervals differed, daily fresh selection caused extreme
turnover, daily tuning was unstable, sentiment was disabled, and historical
membership handling did not match the point-in-time reference. The replacement
design is drafted in docs/design/LIVE_FORWARD_20_SESSION_DRAFT.md.