Determine whether complexity adds value by comparing the portfolio with broad-market, same-universe, risk-based, zero-signal, and randomized-signal alternatives.
- Opportunity benchmark: fully invested SPY.
- Exposure-matched benchmark: 70% SPY plus 30% cash for a 70%-invested strategy.
- Construction controls: point-in-time equal weight and inverse volatility.
- ML attribution controls: zero forecasts and within-date shuffled forecasts.
- Legacy baselines: buy-and-hold, logistic regression, ARIMA, and risk parity.
Same-ten-stock comparisons from the original project are preserved only as historical audits. Those ten present-day winners contain survivor and selection bias and are not credible market benchmarks.
In the one-shot public confirmation:
- Frozen v4 cumulative return: 14.13%; CAGR: 11.34%.
- 70% SPY plus 30% cash cumulative return: 13.77%; CAGR: 11.05%.
- Annualized arithmetic active return: +0.31%.
- Shuffled-score p-value: 0.198.
- 95% active-return interval: -12.79% to +8.93%.
The absolute outcome was slightly higher than the matched benchmark, but the randomized and bootstrap controls did not support a reliable market-beating conclusion.
- Use point-in-time constituents, not today's survivors.
- Match investment exposure and execution costs before attributing security-selection skill.
- Report fully invested SPY separately as practical opportunity cost.
- Keep shuffled forecasts permanent.
- Report confidence intervals, not only point estimates.
- Never replace a failed preregistered benchmark after observing results.