A leakage-aware research platform for cross-sectional stock selection, financial-news NLP, realistic execution, and paper-live monitoring.
The project asks a narrow investment question:
Can machine learning rank future relative winners more reliably than it can predict calibrated stock returns, and does timestamp-valid financial sentiment add useful information?
The current evidence supports continuing that research direction. It does not establish production alpha.
Status date: July 30, 2026
Current experiment: horizon-matched 5/10/20-session paper-live comparison
Universe: point-in-time S&P 500 membership, liquid top 250, sector-capped top 40
Models: Ridge baseline and independently tuned XGBoost challenger
This project did not begin as the current ranking system. It became one by repeatedly challenging attractive backtests and keeping the failed tests.
- We started broad. The original system combined direct return forecasting, CVaR optimization, pair trading, technical features, and financial-news sentiment on a small present-day stock universe. It looked complete, but the architecture was more mature than the evidence. See the original full architecture and experiment trail
- The first audit changed the standard. Fill timing, transaction-cost accounting, label maturity, and optimizer safeguards required correction. That invalidated an early headline result and established the project's fail-closed research discipline. Read the first canonical rerun and corrections
- The question shifted from "what return?" to "which stocks?" Weak return calibration but better cross-sectional ordering led to sector-neutral rank targets, point-in-time membership, nested purged validation, and top-k winner selection. Read the historical rank-40 experiment, later invalidated by the price audit
- Promising additions had to earn activation. Sentiment improved several development point estimates, while smart-universe filtering, survival ML, pair trading, topics, and ensembles either failed gates or remained inconclusive. Negative results were retained rather than blended into an ever-larger strategy. Explore the dated experiment archive
- The live horizon was redesigned, and its historical estimate is being re-audited. The one-session monitor accumulated observations quickly but tested the wrong economic horizon and created extreme turnover. The original matched horizon sweep used the now-invalidated price panel. The active 5/10/20-session comparison remains useful for collecting prospective evidence, but 20 sessions is provisional until the corrected sweep is run. Read the data-integrity audit
- The project is now a focused paper-live experiment. The active system refreshes real market/news data, applies strict FinBERT NLP, independently tunes Ridge and XGBoost, freezes daily forecasts, and tracks separately scheduled 5/10/20-session strategies. Rebalancing policy is the next design question, not an assumed constant.
The long research history remains available through the links above; the rest of this README describes only the final active system and its current evidence.
The July 20 and July 27 tables were invalidated by the price-repair audit. A second audit found that the shared tuning path also reused an inner evaluation fold for tree early stopping. The comparison below replaces those results. All four models use the corrected data, the same seven outer folds, fold-local feature selection, and, for tree models, a separate purged early-stopping split inside each training fold.
The evaluation covers 182 weekly observations from 2020-01-10 through 2023-06-30. Each model arm uses a 70%-invested, sector-capped top-40 portfolio with 8 basis points of one-way variable cost and a $10 fixed fee per trade, assuming $1 million of starting capital. SPY is a full-investment, dividend-adjusted index benchmark on the same next-open holding intervals and does not inherit the strategies' transaction costs.
| Model and portfolio | Model construction | Mean rank IC | Cumulative return | CAGR | Sharpe (net for models) | Maximum drawdown |
|---|---|---|---|---|---|---|
| Ridge | Linear rank model | 0.02058 | 31.83% | 8.22% | 0.443 | -34.29% |
| Ordinary LightGBM | Tree regressor on rank labels | 0.00648 | 2.36% | 0.67% | 0.134 | -34.56% |
| XGBoost ranker | Nonlinear pairwise rank model | 0.01520 | 7.41% | 2.06% | 0.202 | -31.24% |
| LambdaMART | Date-grouped learning-to-rank model | 0.00241 | 24.11% | 6.37% | 0.370 | -38.80% |
| SPY index | Full-investment market benchmark | N/A | 42.89% | 10.73% | 0.583 | -31.62% |
Zero-cost sensitivity is shown separately as an optimistic upper bound. It removes both proportional and fixed execution costs but does not change the holdings or rebalance schedule.
| Portfolio | Zero-cost cumulative return | Zero-cost CAGR | Zero-cost Sharpe | Zero-cost maximum drawdown |
|---|---|---|---|---|
| Ridge | 63.04% | 14.99% | 0.693 | -33.80% |
| Ordinary LightGBM | 31.90% | 8.23% | 0.489 | -33.59% |
| XGBoost ranker | 30.50% | 7.90% | 0.481 | -30.39% |
| LambdaMART | 41.85% | 10.50% | 0.520 | -38.18% |
| SPY index | 42.89% | 10.73% | 0.583 | -31.62% |
Ridge remains the strongest corrected historical model, but it does not beat SPY after the declared costs. Ridge does beat SPY in the zero-cost sensitivity, but that is a research upper bound rather than an implementation claim. LambdaMART turns over less and identifies more eventual top-40 names, but it has lower positive precision, more severe losses, worse CVaR, and a deeper drawdown. None of these development results is a live-return claim, and a future untouched confirmation period is still required.
The Ridge Sharpe of 0.443 is the cost-adjusted value. The same weekly
portfolio returns have a gross Sharpe of 0.693; modeled execution costs
average 0.1168% per week: 0.0462% fixed and 0.0706% proportional. More
capital makes the fixed component smaller, but it does not remove the
proportional component and may increase market impact. Removing only the fixed
fee gives Ridge a 10.85% CAGR and 0.541 Sharpe before any market-impact
allowance. Sharpe values near 1 in older records came from either the
invalidated pre-price-repair panel or a shorter 64-week confirmation window.
The earlier 0.61 corrected-price replay was also superseded by this fully
retuned, common-fold comparison.
Read the framework-clean comparison
The historical system contained many experimental sleeves. Its full architecture is preserved on the archive-experiments branch. The active system keeps only the modules needed to test the current winner-selection and sentiment hypotheses:
flowchart TD
subgraph INGEST["Market, membership, and news ingestion"]
MARKET["Adjusted OHLCV<br/>SPY and sector ETFs<br/>incremental local cache"]
MEMBERSHIP["Effective-dated S&P 500 membership<br/>sector history and corporate events"]
NEWS["Timestamped financial news<br/>persistent deduplication<br/>source and freshness audit"]
end
subgraph PANEL["Leakage-safe research panel"]
UNIVERSE["Lagged 63-session liquidity<br/>point-in-time top 250"]
NLP["Entity linking and FinBERT<br/>event tags and recency<br/>15:45 ET availability clock"]
FEATURES["Technical, residual, risk,<br/>sector-relative, sentiment,<br/>event and recency features"]
TARGETS["Sector-neutral rank targets<br/>5, 10, and 20 sessions<br/>next-open to open"]
end
subgraph LEARN["Horizon-matched model development"]
VALIDATION["Rolling temporal folds<br/>mature labels and horizon purge"]
SELECTION["Fold-local feature ranking<br/>feature-count and Optuna search<br/>XGBoost early stopping"]
MODELS["Independent models<br/>Ridge baseline<br/>XGBoost challenger"]
end
subgraph PORTFOLIO["Portfolio construction and execution"]
SCORES["Score the eligible<br/>cross-section"]
TOP40["Sector-capped top 40<br/>100% research exposure<br/>equal weight"]
CLOCKS["Matched refit and rebalance<br/>5 to 5 / 10 to 10 / 20 to 20"]
TACTICAL["Shadow tactical sentiment event<br/>next-open routing only<br/>same close disabled by default"]
BUFFER["Rebalance buffer<br/>next experiment, not active"]
end
subgraph EVIDENCE["Backtesting, evidence, and live monitoring"]
BACKTEST["Point-in-time walk-forward backtest<br/>costs, turnover, benchmark,<br/>return and drawdown"]
COHORTS["Daily immutable forecasts<br/>partial marks and matched maturity"]
TACTICALLOG["Separate tactical ledger<br/>no credit to strategic returns"]
DIAGNOSTICS["IC, quantiles, distributions,<br/>SHAP and shuffled controls"]
DASHBOARD["Two-page live dashboard<br/>stock-level monitor<br/>horizon comparison"]
end
MARKET --> UNIVERSE
MEMBERSHIP --> UNIVERSE
NEWS --> NLP
MARKET --> FEATURES
UNIVERSE --> FEATURES
NLP --> FEATURES
FEATURES --> TARGETS --> VALIDATION --> SELECTION --> MODELS
FEATURES --> SCORES
MODELS --> SCORES --> TOP40 --> CLOCKS
NLP --> TACTICAL --> TACTICALLOG
BUFFER -. "under audit" .-> TOP40
CLOCKS --> BACKTEST
CLOCKS --> COHORTS
BACKTEST --> DIAGNOSTICS
TACTICALLOG --> DIAGNOSTICS
COHORTS --> DIAGNOSTICS --> DASHBOARD
| Module | Active code | Responsibility |
|---|---|---|
| Market, membership, and news ingestion | src/data, src/nlp |
Refresh adjusted market history and financial news; retain effective-dated membership and sectors; deduplicate records; preserve source, coverage, timestamp, and freshness audits |
| Leakage-safe research panel | src/features, src/targets |
Form the lagged liquid universe; run strict local FinBERT and event tagging; build technical, cross-sectional, residual, risk, sentiment, and recency features; create horizon-matched rank targets |
| Horizon-matched model development | src/models/rank_models.py, src/experiments/live_forward_test.py |
Purge overlapping labels; rank features inside temporal folds; tune feature count and hyperparameters; apply early stopping; keep Ridge and XGBoost selection independent |
| Portfolio construction and execution | src/portfolio/sparse_rank.py, src/experiments/live_forward_20d.py |
Score the eligible top 250; enforce the sector cap; freeze equal-weight top 40; operate matched refit/rebalance clocks; route qualifying tactical events to the next open; keep same-close routing disabled unless quote and broker-cutoff controls are supplied |
| Backtesting, evidence, and live monitoring | src/backtest, src/experiments |
Simulate next-open execution and costs; mature immutable cohorts; keep tactical attribution separate from strategic returns; report return, IC, turnover, drawdown, distributions, SHAP, and shuffled controls; publish the two-page dashboard |
| Finding | Current evidence | Boundary |
|---|---|---|
| Winner selection is more promising than raw-return prediction | Framework-clean Ridge rank IC 0.02058 and sector-capped top-40 excess 0.2227% per weekly observation |
Average signal is thin, unstable by year, and still requires a future untouched confirmation period |
| Sentiment appears useful | Technical-plus-sentiment point estimates exceeded technical-only at 5, 10, and 20 sessions | The comparison does not yet prove a causal NLP increment; prior robustness gates were inconclusive |
| The best horizon is unresolved | The original 5/10/20-session sweep was invalidated by the July 30 price-repair audit | Keep collecting all three prospective arms and rerun the matched historical sweep on corrected data |
| Rebalancing behavior is model-specific | Over six July 20-28 sessions, freezing improved Ridge net return by 3.93 points; daily replacement improved XGBoost by only 0.53 points net |
Six sessions cannot determine permanent policy; a preregistered buffer comparison is required |
| Negative selected stocks persist, while severe losses cluster in stressed markets | Ridge sign false positives occur in every half-year; 2020 H1 and 2022 H1 contain 70.43% of severe observations and 75.79% of severe-loss drag |
The realized market state is explanatory, not available at decision time; the causal SPY regime does not reliably identify losing weeks |
| A bounded tail veto can reduce direct severe-loss drag | The fully invested historical A/B treatment reduced severe-loss contribution by about 8.1% relative, with a paired interval entirely below zero |
CVaR improvement remained uncertain, return non-inferiority failed its interval, active arms required a 30% veto budget, and many rejected stocks were winners |
| Live operation is feasible | Current market/news refresh, strict local FinBERT, purged tuning, frozen forecasts, and immutable manifests completed | The redesigned live forecasts have not matured; there is no live-return claim yet |
Detailed decision records:
- Framework-clean four-model comparison
- Live forecast-horizon evidence
- Backtest tail-risk timeline
- Severe-loss risk-control A/B experiment
This development backtest compares Ridge, ordinary LightGBM, XGBoost, and LambdaMART on identical corrected outer folds after the declared research costs. The growth panel always includes full-investment SPY as the market index, using dividend-adjusted next-open returns without the strategy's portfolio costs. The lower panels focus on Ridge and LambdaMART drawdown and return distribution, while turnover is shown for all four models. Cumulative return alone can hide left-tail risk and implementation drag. Ridge remains the control; LambdaMART did not qualify for promotion.
Source: Framework-clean rank-model comparison.
The score report uses the framework-clean Ridge OOS scores and separates prediction quality from portfolio accounting:
- score-quintile cumulative returns;
- same-date Spearman information coefficient;
- positive-IC frequency;
- top-40 versus universe return distribution; and
- average top-40 excess return.
The quintile paths, weekly IC variation, and return overlap are important warnings. A positive average rank IC does not automatically imply a stable or low-risk portfolio.
Source: Framework-clean rank-model comparison.
The chart uses XGBoost's exact tree contributions for the frozen July 30 20-session model across its 249-name eligible cross-section. It is a local model explanation, not causal importance. Features selected for training can still have zero SHAP contribution when the fitted trees never split on them.
Regenerate all README figures from the local research artifacts:
python -m src.experiments.readme_figuresThe SVGs are tracked; raw datasets, models, and generated report directories remain excluded from Git.
The active experiment issues new forecasts every trading day while matching each strategy's model-refit and portfolio-rebalance clock to its horizon:
| Arm | Model refit | Portfolio rebalance | Label purge | Authority |
|---|---|---|---|---|
| Ridge / XGBoost 5-session | Every 5 sessions | Every 5 sessions | 5 sessions | Short-horizon diagnostic |
| Ridge / XGBoost 10-session | Every 10 sessions | Every 10 sessions | 10 sessions | Required challenger |
| Ridge / XGBoost 20-session | Every 20 sessions | Every 20 sessions | 20 sessions | Primary research horizon |
Every scheduled refit performs:
- matured-label filtering;
- rolling temporal validation;
- horizon-matched purging;
- fold-local feature ranking and feature-count selection;
- independent Ridge/XGBoost hyperparameter search;
- final refit on eligible historical labels; and
- immutable model, feature, score, and top-40 artifacts.
Daily forecasts accumulate observations without forcing daily portfolio replacement. After the initial horizon delay, approximately one forecast per horizon matures each trading day, but overlapping outcomes require HAC or block-bootstrap inference.
Run one completed-market-day cycle:
python -m src.cli run-live-forward-20d --as-of YYYY-MM-DDThe default v2 output is reports/live_forward_horizon_matched_v2/. The first
shared-clock implementation remains frozen under reports/live_forward_20d/
and is not mixed into the corrected history.
See the 20-session live-forward design for execution clocks, tactical sentiment gates, evidence thresholds, and artifacts.
The legacy one-session monitor provides a small but useful out-of-sample ablation. It compares the actual daily-changing top 40 with the names frozen from July 20:
| Model | Daily-changing top 40 | Keep July 20 stocks | Better choice |
|---|---|---|---|
| Ridge gross | -1.46% | +1.86% | Freeze by 3.32 points |
| Ridge net | -2.19% | +1.74% | Freeze by 3.93 points |
| XGBoost gross | -0.58% | -1.78% | Daily by 1.20 points |
| XGBoost net | -1.37% | -1.90% | Daily by 0.53 points |
| SPY | -0.17% | -0.17% | N/A |
"Net" uses $1 million starting capital, 8 bps one-way execution cost, and $10 per trade. Daily selection generated approximately 616% Ridge turnover and 662% XGBoost turnover including initial purchases; frozen-name portfolios were near 100%. Frozen equal-weight and buy-and-hold controls were very similar, so membership changes, rather than small weight drift, drove most of the difference.
This result supports removing daily full replacement, but it does not prove that every scheduled rebalance should be a complete top-40 refresh. The current 5/10/20-session design therefore remains under audit. The next controlled experiment will compare, separately for Ridge and XGBoost:
- full horizon-matched refresh;
- frozen holdings;
- rank-buffer replacement; and
- score-margin replacement.
Buffer thresholds must be selected on historical temporal validation and frozen before forward use. The six observed sessions will not choose the threshold.
Read the complete rebalancing ablation
Review the rebalance-buffer experiment draft
The legacy one-session experiment is now a frozen-cohort monitoring track rather than a source of new daily portfolios. Its daily command refreshes prices, extends the existing July 20-27 Ridge/XGBoost cohorts, rebuilds the local dashboard, and regenerates the sanitized GitHub Pages bundle:
python -m src.cli run-live-forward-daily --as-of YYYY-MM-DDThe public monitor also refreshes automatically at 22:30 UTC on weekdays. The scheduled job rebuilds the same frozen-cohort paths from sanitized selections, fails closed on missing prices, validates the bundle, and deploys the refreshed site. The default Portfolio vs Benchmark chart now follows this monitoring path, so its latest date advances even though stock selection remains frozen.
The Holding-period laboratory can keep any dated list unchanged or compare custom 1-60-session replacement horizons, including 5/10/20-session presets. Ridge and XGBoost can be toggled independently while SPY remains permanently visible from the shared decision-close baseline. It reports net return, SPY-relative return, replacement-only turnover, and rebalance counts under identical execution assumptions.
The GitHub Pages workflow publishes two linked pages with one shared visual
shell from main:
The original Stock-level view remains canonical rather than being duplicated on the horizon page. The two-page contract and evidence boundaries are recorded in the dashboard merge design.
This monitoring track does not replace the primary horizon-matched 5/10/20
experiment described above. The former one-session fitting workflow remains
available only through the explicit --retrain flag and is not part of the
normal monitoring run.
The main pipeline is designed to fail closed:
- Historical membership and liquidity are reconstructed point in time.
- Sector classifications are joined backward by effective date.
- Features must exist before the declared decision timestamp.
- Labels must mature before training and are purged from validation boundaries.
- Feature selection and tuning occur inside temporal training folds.
- Ridge remains the hard-to-fake baseline.
- Historical sentiment comparisons require a technical-only paired arm. The current six-arm paper-live monitor includes sentiment in every model, so it cannot by itself attribute future performance specifically to sentiment.
- Same-close news execution requires timestamped quotes and a verified broker cutoff; otherwise execution moves to the next open.
- Missing real market data, stale news, failed FinBERT inference, constant model scores, or incomplete artifacts stop the run.
- Gross selection evidence, cost-adjusted performance, and tactical attribution are reported separately.
python -m venv .venv
# Windows
.venv\Scripts\activate
pip install -r requirements.txt
pip install -e ".[tuning]"
pytest -qUseful research commands:
# Compare matched 5/10/20-session horizons
python -m src.cli run-forecast-horizon-sweep
# Run one paper-live cycle after a completed US market close
python -m src.cli run-live-forward-20d --as-of YYYY-MM-DDRun python -m src.cli --help for the complete command surface.
The main branch keeps the active research system and its current evidence
contract concise. Superseded designs, failed gates, and the full experiment
trail remain valuable but should not be mistaken for simultaneously active
strategies.
- Current documentation index
- Project status and evidence map
- Methodology equations
- Current live-forward design
- Rebalance-buffer experiment draft
- Live rebalancing ablation
- Historical experiments and superseded designs
- Archived branch README
Failed experiments are retained as evidence. They are not removed merely because they weaken the preferred narrative.
The next risk-control version should not increase the current treatment's authority. The first historical A/B result is inconclusive and remains disabled. Future work will:
- add decision-time overnight-gap, recent-drawdown, liquidity-deterioration, earnings-proximity, SPY-stress, and timestamp-safe event features;
- compare transparent rules, class-weighted models, focal-style loss, and conditional quantiles inside purged temporal folds;
- target fewer, higher-confidence removals instead of depending on a 30% veto budget;
- preserve the fully invested 40-stock Ridge control so cash exposure cannot create an artificial risk improvement;
- optimize severe-loss precision and recall together with rejected-winner cost, CVaR, drawdown, turnover, and return non-inferiority;
- evaluate stability separately across stressed and ordinary periods using only causal regime inputs;
- extend point-in-time testing beyond the current three-and-a-half-year portfolio history; and
- start a prospective shadow A/B test only after a frozen historical treatment passes both paired risk improvement and return non-inferiority.
The preregistered workflow and failure history are maintained in the risk-control A/B plan and false-positive research record.
- No redesigned 20-session paper-live forecast has matured yet.
- The development backtest reported above contains 182 OOS weeks from 2020-01-10 through 2023-06-30; the underlying data and training history begin in 2018. The evaluated portfolio period is therefore about three and a half years, not a full decade, and spans too few independent regimes for a durable claim. Extending point-in-time testing toward at least ten years is a research target, not a guarantee of adequacy: long-window backtests still require regime analysis, untouched confirmation, and correction for the number of strategies tried. See CFA Institute's backtesting guidance and the deflated-Sharpe discussion of sample length and multiple testing.
- The strongest portfolio results remain development evidence, not a genuine untouched confirmation.
- Public data does not fully solve delisted-security coverage or historical sector classification.
- The average ranking edge is small and varies materially by year and regime.
- Sentiment is promising, but coverage, timestamp quality, and causal attribution remain incomplete. The current six live model/horizon arms do not include a technical-only prospective control.
- The high-recall survival model has not passed its activation gates and remains a shadow watchlist.
- Same-close tactical execution is unvalidated without intraday quotes and actual paper or broker fills.
- Scheduled full top-40 replacement remains under audit; the six-session ablation is too short to activate a freeze or buffer.
- Static cost assumptions do not establish capacity, taxes, borrow availability, market impact, or partial-fill behavior.
- The first fully invested severe-loss A/B treatment reduced direct tail drag but did not establish CVaR improvement or return non-inferiority. It remains inactive.
This repository is an educational and research system. Backtests, model scores, paper portfolios, and live-forward monitors are not investment advice and are not evidence of guaranteed future performance.