Audit price integrity and tail-risk controls

Author: HanSun103

README.md

@@ -50,11 +50,13 @@ repeatedly challenging attractive backtests and keeping the failed tests.
    inconclusive. Negative results were retained rather than blended into an
    ever-larger strategy.
    [Explore the dated experiment archive](https://github.com/HanSun103/AI-Driven-Portfolio-Optimization-Backtesting-System/tree/archive-experiments/docs/experiments)
-5. **The live horizon was redesigned.** The one-session monitor accumulated
-   observations quickly but tested the wrong economic horizon and created
-   extreme turnover. A matched historical sweep identified a 10-to-20-session
-   plateau, with 20 sessions as the leading, but not uniquely proven, candidate.
-   [Read the horizon decision](docs/experiments/2026-07-28-live-horizon-decision.md)
+5. **The live horizon was redesigned, and its historical estimate is being
+   re-audited.** The one-session monitor accumulated observations quickly but
+   tested the wrong economic horizon and created extreme turnover. The original
+   matched horizon sweep used the now-invalidated price panel. The active
+   5/10/20-session comparison remains useful for collecting prospective
+   evidence, but 20 sessions is provisional until the corrected sweep is run.
+   [Read the data-integrity audit](docs/experiments/2026-07-30-price-repair-oos-audit.md)
 6. **The project is now a focused paper-live experiment.** The active system
    refreshes real market/news data, applies strict FinBERT NLP, independently
    tunes Ridge and XGBoost, freezes daily forecasts, and tracks separately
@@ -64,26 +66,29 @@ repeatedly challenging attractive backtests and keeping the failed tests.
 The long research history remains available through the links above; the rest
 of this README describes only the final active system and its current evidence.
 
-### Ridge, XGBoost, and ensemble development backtest
+### Corrected Ridge, XGBoost, and ensemble development backtest
 
-This is the complete portfolio comparison used by the current README
-visualization. It covers 182 weekly observations from 2020-01-10 through
-2023-06-30. Each model holds 40 equal-weight stocks with a 25% sector cap.
-Results include 8 basis points per one-way traded weight and a $10 fixed fee
-per trade.
+An audit on July 30 found that the historical price-repair merge could mix
+split-adjusted and unadjusted rows with the same index. The July 20 and July 27
+tables were invalidated. The table below is the corrected replay after rebuilding
+prices, sectors, labels, current features, and Alpha158 factors, then refitting
+the saved fold specifications. It covers 182 weekly observations from
+2020-01-10 through 2023-06-30. Each model holds 40 equal-weight stocks with a
+25% sector cap. Results include 8 basis points per one-way traded weight and a
+$10 fixed fee per trade.
 
 | Model and portfolio | Model construction | Mean rank IC | Net cumulative return | Net CAGR | Sharpe | Maximum drawdown |
 |---|---|---:|---:|---:|---:|---:|
-| Ridge | Linear rank model | 0.00560 | 103.88% | 22.57% | 0.77 | -27.37% |
-| XGBoost ranker | Nonlinear pairwise rank model | 0.00222 | 190.28% | 35.59% | 0.97 | -39.44% |
-| Ridge-XGBoost ensemble | Expanding prior-fold blend; 50/50 in the first fold, then 25% Ridge and 75% XGBoost | 0.00188 | 110.05% | 23.62% | 0.78 | -35.92% |
+| Ridge | Linear rank model | 0.02027 | 66.96% | 15.77% | 0.61 | -40.59% |
+| XGBoost ranker | Nonlinear pairwise rank model | 0.00724 | 2.11% | 0.60% | 0.17 | -41.21% |
+| Ridge-XGBoost ensemble | Expanding prior-fold blend; historical selection now favors Ridge | 0.01805 | 49.68% | 12.21% | 0.52 | -37.71% |
 | SPY benchmark | Buy and hold with the experiment's 0.5% annual management fee | N/A | 39.58% | 10.00% | 0.54 | -34.12% |
 
-The ensemble improved the top-40 excess-return point estimate over Ridge but
-did not beat XGBoost. Its active-return confidence interval crossed zero and
-its shuffled-score information-ratio p-value was `0.089`, so the experiment
-did not establish a promotion claim.
-[Read the Ridge-XGBoost rank ensemble experiment](docs/experiments/2026-07-27-ridge-xgb-rank-ensemble.md)
+Ridge is the strongest corrected historical model. XGBoost no longer supports
+the earlier claim of stronger top-tail selection. The active-return confidence
+intervals still cross zero, and the final chronological fold trails SPY, so
+this remains development evidence rather than a promotion claim.
+[Read the price-repair and OOS evidence audit](docs/experiments/2026-07-30-price-repair-oos-audit.md)
 
 ## Architecture
 
@@ -161,14 +166,19 @@ flowchart TD
 
 | Finding | Current evidence | Boundary |
 |---|---|---|
-| Winner selection is more promising than raw-return prediction | Repaired weekly OOS rank IC `0.0059`; top-40 excess `0.1686%` per week; shuffled-score p-value `0.0099` | Average signal is thin, unstable by year, and the active-return confidence interval crosses zero |
+| Winner selection is more promising than raw-return prediction | Corrected Ridge rank IC `0.02027` and top-40 excess `0.2146%` per weekly observation | Average signal is thin, unstable by year, and the active-return confidence interval crosses zero |
 | Sentiment appears useful | Technical-plus-sentiment point estimates exceeded technical-only at 5, 10, and 20 sessions | The comparison does not yet prove a causal NLP increment; prior robustness gates were inconclusive |
-| Twenty sessions is the leading horizon | Full-OOS annualized active point estimates: 5d `4.53%`, 10d `6.65%`, 20d `8.59%` | Twenty versus ten sessions was not statistically distinguishable (`t = 0.33`) |
+| The best horizon is unresolved | The original 5/10/20-session sweep was invalidated by the July 30 price-repair audit | Keep collecting all three prospective arms and rerun the matched historical sweep on corrected data |
 | Rebalancing behavior is model-specific | Over six July 20-28 sessions, freezing improved Ridge net return by `3.93` points; daily replacement improved XGBoost by only `0.53` points net | Six sessions cannot determine permanent policy; a preregistered buffer comparison is required |
+| Negative selected stocks persist, while severe losses cluster in stressed markets | Ridge sign false positives occur in every half-year; 2020 H1 and 2022 H1 contain `70.43%` of severe observations and `75.79%` of severe-loss drag | The realized market state is explanatory, not available at decision time; the causal SPY regime does not reliably identify losing weeks |
+| A bounded tail veto can reduce direct severe-loss drag | The fully invested historical A/B treatment reduced severe-loss contribution by about `8.1%` relative, with a paired interval entirely below zero | CVaR improvement remained uncertain, return non-inferiority failed its interval, active arms required a `30%` veto budget, and many rejected stocks were winners |
 | Live operation is feasible | Current market/news refresh, strict local FinBERT, purged tuning, frozen forecasts, and immutable manifests completed | The redesigned live forecasts have not matured; there is no live-return claim yet |
 
-The detailed decision record is
-[Live forecast-horizon evidence](docs/experiments/2026-07-28-live-horizon-decision.md).
+Detailed decision records:
+
+- [Live forecast-horizon evidence](docs/experiments/2026-07-28-live-horizon-decision.md)
+- [Backtest tail-risk timeline](docs/experiments/2026-07-30-backtest-tail-risk-timeline.md)
+- [Severe-loss risk-control A/B experiment](docs/experiments/2026-07-30-severe-loss-risk-control-ab.md)
 
 ## Visual evidence
 
@@ -416,6 +426,33 @@ strategies.
 Failed experiments are retained as evidence. They are not removed merely
 because they weaken the preferred narrative.
 
+## Future work
+
+The next risk-control version should not increase the current treatment's
+authority. The first historical A/B result is inconclusive and remains disabled.
+Future work will:
+
+1. add decision-time overnight-gap, recent-drawdown, liquidity-deterioration,
+   earnings-proximity, SPY-stress, and timestamp-safe event features;
+2. compare transparent rules, class-weighted models, focal-style loss, and
+   conditional quantiles inside purged temporal folds;
+3. target fewer, higher-confidence removals instead of depending on a 30%
+   veto budget;
+4. preserve the fully invested 40-stock Ridge control so cash exposure cannot
+   create an artificial risk improvement;
+5. optimize severe-loss precision and recall together with rejected-winner
+   cost, CVaR, drawdown, turnover, and return non-inferiority;
+6. evaluate stability separately across stressed and ordinary periods using
+   only causal regime inputs;
+7. extend point-in-time testing beyond the current three-and-a-half-year
+   portfolio history; and
+8. start a prospective shadow A/B test only after a frozen historical treatment
+   passes both paired risk improvement and return non-inferiority.
+
+The preregistered workflow and failure history are maintained in the
+[risk-control A/B plan](docs/design/SEVERE_LOSS_RISK_CONTROL_AB_PLAN.md) and
+[false-positive research record](docs/research/2026-07-30-false-positive-and-tail-risk-control.md).
+
 ## Known limitations
 
 1. No redesigned 20-session paper-live forecast has matured yet.
@@ -445,7 +482,10 @@ because they weaken the preferred narrative.
 9. Scheduled full top-40 replacement remains under audit; the six-session
    ablation is too short to activate a freeze or buffer.
 10. Static cost assumptions do not establish capacity, taxes, borrow
-   availability, market impact, or partial-fill behavior.
+    availability, market impact, or partial-fill behavior.
+11. The first fully invested severe-loss A/B treatment reduced direct tail
+    drag but did not establish CVaR improvement or return non-inferiority. It
+    remains inactive.
 
 ## Research disclaimer
 

configs/experiments/live_forward_20d.yaml

@@ -21,6 +21,8 @@ live_forward_20d:
       - rank_margin_buffer
       - score_percentile_buffer
   training_lookback_sessions: 756
+  # Increment after a historical data correction that invalidates fitted models.
+  training_data_revision: 2
   holdings: 40
   invested_weight: 1.0
   maximum_sector_fraction: 0.25

docs/FINAL_VERSION.md

@@ -1,15 +1,19 @@
 # Final research-reference version
 
+> **Evidence correction, July 30, 2026:** performance estimates built from the
+> pre-correction repaired price panel are invalid. The design record remains
+> useful, but its quantitative claims must not be cited. See the
+> [price-repair and OOS evidence audit](experiments/2026-07-30-price-repair-oos-audit.md).
+
 Status: built and verified; not validated live alpha
 
 ## Current note
 
 This final-reference package remains the conservative fail-closed build from the earlier frozen
-v4 contract. A later repaired development run found stronger evidence for the full weekly
-rank-40 ML stock-selection direction: 37.22% net CAGR, 0.99 Sharpe, and -22.77% maximum drawdown
-from 2020-04-24 through 2023-06-30. That newer result is promising but still development-only.
-It should be treated as the next candidate direction to freeze and confirm, not as a replacement
-for a passed final alpha claim.
+v4 contract. The corrected July 30 development replay identifies Ridge as the strongest historical
+rank-40 candidate, with 15.77% net CAGR, 0.61 Sharpe, and -40.59% maximum drawdown from 2020
+through 2023 H1. Its active-return confidence interval crosses zero and its final chronological
+fold trails SPY. It is a candidate to test prospectively, not a passed alpha claim.
 
 ## Purpose