Skip to content

NEXT_RESEARCH_DIRECTION.md

Latest commit

 

History

History
217 lines (165 loc) · 10.6 KB

NEXT_RESEARCH_DIRECTION.md

File metadata and controls

217 lines (165 loc) · 10.6 KB
Raw

Research roadmap after v4 confirmation

Status: historical roadmap, partially executed. The point-in-time NMF, exact-time event, strict weekly NLP, and pair experiments are complete. Their evidence-backed disposition is recorded in the final system proposal. BERTopic, structured LLM events, selector changes, and combined portfolio changes remain unimplemented research ideas rather than parts of the final reference system.

Motivation

The frozen v4 confirmation produced a small positive rank IC and slightly higher net return than the exposure-matched SPY benchmark, but it failed the shuffled-score and confidence-interval gates. This does not prove that statistical ML is useless. It shows that the current ranking signal was not distinguishable from chance over the 64-week confirmation sample.

The next version should therefore stop asking one model to discover the whole portfolio from noisy price features. The proposed system uses transparent stock selection to narrow the point-in-time S&P 500, NLP/LLM methods to structure forward-looking information, traditional ML only as a conservative signal combiner, and independent pair research for diversification.

Proposed allocation of research effort

Workstream Approximate focus Purpose
Data, backtesting, and controls 30% Make every conclusion reproducible and difficult to fake
Sentiment, topics, and LLM events 30% Extract forward-looking information from text
Transparent stock selection 25% Narrow the S&P 500 with explainable signals
Pair trading 15% Develop an independently validated market-neutral sleeve

These percentages are planning guidance, not portfolio weights.

Proposed selection funnel

flowchart LR
    PIT["Point-in-time S&P 500"] --> LIQ["Liquidity and data-quality screen<br/>approximately 300-400 names"]
    LIQ --> FACTOR["Transparent factor and relative-strength rank<br/>approximately 75-100 names"]
    FACTOR --> TEXT["Sentiment, topic, and structured LLM events<br/>approximately 30-50 names"]
    TEXT --> PORT["Hold zone and diversification<br/>final 30-40 names"]
Loading

The counts are candidate ranges to evaluate inside nested development, not values to choose after observing confirmation results.

Workstream A: transparent stock selection

The first selector should be reproducible without AI:

  • sector-neutral residual momentum;
  • medium-term relative strength;
  • profitability and balance-sheet quality when point-in-time fundamentals are available;
  • earnings surprise and analyst revisions when historically timestamped data are available;
  • volatility, drawdown stability, liquidity, and transaction-cost penalties;
  • sector and concentration controls.

Run an equal-weight top-K bake-off for 20, 30, 40, and 50 holdings. Compare residual momentum, quality plus momentum, sector-neutral relative strength, low-volatility quality, the frozen Ridge rank, and randomized selection. K selection must occur inside nested validation.

Workstream B: sentiment, topic modeling, and LLM events

Current capability

The repository currently has FinBERT sentiment, sentence embeddings for ticker relevance and semantic search, fixed event categories, and point-in-time NMF topic features. It does not currently have enough exact-date coverage for LDA model selection, a completed BERTopic run, dynamic embedding-topic tracking, or topic-regime allocation. An LDA adapter exists, but both batch and bounded online full monthly runs exceeded the fixed 15-minute ceiling.

Proposed topic layer

Use embedding-based topic discovery as the primary candidate because financial headlines are short, vocabulary changes, and topics evolve. NMF and LDA should remain transparent baselines.

Reusable prior-work framework

The author's earlier MMAI 891 topic-modeling notebook provides a useful starting experiment. It compared NMF and LDA for K = [8, 10, 12, 15], used TF-IDF for NMF and count vectors for LDA, measured c_v coherence and topic-word diversity, and inspected representative documents. On 14,748 hotel/restaurant reviews, NMF with 10 topics had the best coherence (0.5083) and topic diversity of 0.69.

We should reuse the evaluation pattern, not its fitted result:

  • compare multiple topic families and topic counts;
  • measure coherence and diversity together;
  • inspect top terms and representative articles before assigning topic labels;
  • retain deterministic seeds and fitted model artifacts;
  • record why one configuration was chosen.

The hotel-review value K=10 is not a prior for financial news. Reviews are longer and their themes are relatively stable; headlines are shorter, event driven, time varying, and linked to multiple entities. Financial preprocessing must also retain meaningful numbers, ticker symbols, negation, acronyms, and multiword phrases that the earlier general-purpose cleaning removed.

For each point-in-time news window, the proposed topic layer would produce:

  • topic_id and human-readable topic label;
  • macro, sector, company, supply-chain, competitive, or regulatory scope;
  • topic novelty versus recent history;
  • topic volume and acceleration;
  • cross-source dissemination breadth;
  • topic sentiment and disagreement;
  • affected tickers, sectors, suppliers, customers, competitors, and candidate pairs;
  • topic persistence and decay;
  • model version, clustering window, and data cutoff.

Topics must be fitted or updated using only text available by that date. A topic model fitted once on the full archive would leak future vocabulary and event structure.

Topic configuration must not be selected by coherence alone. The finance experiment should also measure rolling topic stability, out-of-sample document coverage, topic turnover, human label consistency, and incremental rank IC/portfolio value after costs. A coherent topic that provides no forward-looking or risk-control information is descriptive, not an investment signal.

Proposed LLM role

The LLM should be a structured information extractor, not an unconstrained stock picker. Each event should return a versioned schema such as:

{
  "event_type": "earnings_guidance",
  "scope": "company",
  "direction": "negative",
  "surprise": 0.72,
  "novelty": 0.81,
  "expected_horizon": "one_to_four_weeks",
  "confidence": 0.77,
  "affected_tickers": ["AAPL", "QCOM", "SWKS"],
  "evidence": ["source sentence or document identifier"]
}

The portfolio engine—not the LLM—must translate these fields into ranks and weights. Outputs must be cached with prompt, model version, source text hash, timestamp, and evidence.

Required NLP controls

Compare under identical universes, holdings, costs, and dates:

  1. transparent selector only;
  2. selector plus FinBERT;
  3. selector plus discovered topics;
  4. selector plus structured LLM events;
  5. selector plus timestamp-shuffled text signals;
  6. an LLM-only selector as a diagnostic, not the default architecture.

Slow weekly text state and exact-timestamp tactical events remain separate experiments.

Workstream C: supporting ML

Traditional ML remains useful for a narrower job:

  • combine structured numeric, sentiment, topic, and event signals;
  • learn modest nonlinear interactions;
  • estimate whether a signal is reliable in the current regime;
  • shrink weak predictions toward zero;
  • rank candidates rather than emit unconstrained expected returns.

Regularized linear ranking remains the minimum baseline. A more complex model must beat it in nested OOS tests and permanent shuffled-signal controls.

Workstream D: pair trading

Pair trading is a separate market-neutral research track, not a way to repair failed directional confirmation. It should require:

  • point-in-time candidates and positive economic relationships;
  • cointegration rather than correlation alone;
  • rolling hedge-ratio and half-life stability;
  • borrow, short availability, dividends, spread, impact, and forced-exit costs;
  • randomized-pair and correlation-only controls;
  • topic/event break rules that suspend convergence assumptions after structural news;
  • low correlation with the directional sleeve.

Its portfolio weight remains zero until the pair contract passes its own preregistered gates and the combined portfolio has a separate untouched evaluation.

The first v5 implementation is now observed. It passed timing, drawdown, turnover, capacity, and positive-point-estimate gates, but failed the positive confidence bound and random-pair placebo. Its +0.038% annualized increment moved core CAGR by only about 4.5 basis points. This version therefore remains at zero weight and must not be retuned on the same sample.

Experiment sequence

  1. Selector bake-off: identify whether transparent ranking can create a stable 30-40-name portfolio before adding NLP.
  2. Topic baseline reproduction: adapt the earlier NMF/LDA grid, coherence, diversity, and representative-document review to financial news without using future vocabulary.
  3. Dynamic topic experiment: measure whether point-in-time embedding topic state adds information beyond NMF/LDA, FinBERT, and topic-shuffled controls.
  4. Structured LLM experiment: test incremental value beyond the frozen selector and FinBERT.
  5. Independent pair experiment: test tradability and diversification without directional portfolio allocation.
  6. Combined contract: freeze only modules that passed independently, then wait for or acquire a genuinely untouched period.

The observed 2024 through 2025 Q1 confirmation period may be used for diagnosis but never again described as untouched confirmation.

Primary success measures

  • Stock selection: net information ratio versus the transparent equal-weight selector.
  • Forecasting: weekly rank IC and stability by regime.
  • NLP/LLM: incremental return and IC versus FinBERT, plus timestamp-shuffled placebo results.
  • Pair trading: net Sharpe, drawdown, capacity, and correlation with the directional sleeve.
  • Combined portfolio: net active return versus exposure-matched SPY, block-bootstrap interval, shuffled/permutation distribution, turnover, cost, and drawdown.

Research references