Skip to content

README.md

Latest commit

 

History

History
116 lines (95 loc) · 6.61 KB

README.md

File metadata and controls

116 lines (95 loc) · 6.61 KB
Raw

Sentiment and event NLP module

Goal

Turn ticker-linked financial text into leakage-safe sentiment and event signals. The module supports two distinct uses: slow sentiment state for weekly ranking and, only when exact timestamps exist, tactical event triggers for next-open execution.

How it works

  1. source_connectors.py collects enabled RSS, NewsAPI, Alpha Vantage, Reddit, and X sources. Reddit and X are optional.
  2. entity_linking.py associates text with securities.
  3. sentiment_model.py scores text with FinBERT. Confirmation is configured to error rather than fall back to a lexical model.
  4. event_detector.py tags earnings, regulation, policy, product, litigation, and related events.
  5. nlp_features.py creates 1-, 3-, and 7-day sentiment, mentions, mention spikes, event novelty, and recency-decay features.
  6. topic_model.py creates point-in-time NMF and LDA topic features and supports BERTopic as an optional backend.
  7. FeatureStore.join_nlp_features shifts ticker features by one business day before use.

Sentence embeddings remain available for ticker relevance and semantic search. NMF and LDA topic discovery are implemented separately; BERTopic requires its optional package.

Date-only weekend news is rolled to the next business day and then receives the normal one-business-day lag. It is never treated as known before its actual availability.

Current achievement and evidence

  • Confirmation FinBERT run scored 65,024 unique texts and mapped them to 79,364 ticker rows.
  • Eight ticker-day NLP features were generated for 247 securities.
  • No lexical fallback was used in confirmation.
  • In sparse-v3 development, allowing slow sentiment increased CAGR from 5.47% to 10.98%.
  • Development incremental return was +6.14% annualized; its 90% block-bootstrap interval was +0.65% to +12.82%, while the 95% interval still crossed zero at -0.23% to +14.15%.
  • The expanded v5 NMF baseline completed 71 point-in-time monthly refits on 235,993 FNSPID posts and generated 16 raw topic features.
  • Exact-date eligible coverage is 52,588 rows (13.95%) across 313 tickers. The predeclared 50% coverage gate failed, so no topic uplift claim has been made.
  • NMF median rolling alignment is 0.905 and median topic diversity is 0.91. Full monthly batch and bounded online LDA both exceeded the fixed 15-minute runtime ceiling.
  • The exact-time development event study matched 935 direct-ticker FinBERT signals across 62 2020 entry dates. Five-session net event return was -43.72 basis points and incremental return versus matched no-news controls was -44.62 basis points; timing and sign placebos both failed.
  • A stricter weekly test held the numeric OOS forecast and portfolio fixed. Sentiment plus topics improved CAGR from 7.22% to 9.10%, but the +1.71% annualized increment had a 95% interval of -1.89% to +5.31% and failed the timestamp-shuffle control (p=0.366). Combined row coverage was only 12.71%, so this is not confirmed NLP alpha.

That A/B result is encouraging development evidence, not proof that sentiment caused the entire uplift. The arms could select different features, model parameters, and holdings.

Limitations

  • NLP coverage is not complete across the point-in-time universe.
  • Most FNSPID development records remain date-only, which prevents a tactical intraday claim.
  • URL slugs or first sentences are used as audited headline proxies when titles are unavailable.
  • Macro-, industry-, and company-level sentiment are not yet modeled as separate propagation channels.
  • Tactical sentiment remains disabled for the broad archive. The small verified-time subset did not pass its data-purchase or trading-value gate.
  • LDA is implemented but did not finish the full monthly contract within 15 minutes. A real BERTopic development run remains gated on broader usable coverage.

Topic-modeling layer

The NMF point-in-time baseline and an LDA adapter are implemented. BERTopic remains the preferred semantic candidate after the coverage gate is repaired. The complete layer should distinguish macro, sector, company, supply-chain, competitive, and regulatory narratives; measure novelty, volume acceleration, dissemination, sentiment, and persistence; and connect each topic to affected securities and possible pair breaks.

A topic model must be trained or updated using only text available at each historical decision date. Fitting clusters on the complete archive would leak future vocabulary and event structure. See the next research direction for its proposed schema and evaluation controls.

An earlier MMAI 891 notebook already supplies a reusable baseline pattern: NMF versus LDA across several K values, c_v coherence, topic diversity, top words, and representative documents. Its best hotel-review result was NMF with K=10, coherence 0.5083, and diversity 0.69. Those values are not financial-news parameters; only the comparison and interpretability workflow will be reused. Finance-specific evaluation must add rolling stability, OOS coverage, temporal leakage checks, and incremental investment value after costs.

Main commands

python -m src.cli build-nlp-features --help
python -m src.cli build-confirmation-nlp-features --help
python -m src.cli build-topic-features --help
python -m src.cli augment-residual-rank-topics --help
python -m src.cli run-timestamped-news-event-study --help
python -m src.cli run-weekly-nlp-incremental-v5 --help
python -m src.cli run-news-ablation --help
python -m src.cli run-sparse-sentiment-ablation --help

Improvement backlog

  • Separate macro, sector, company, and pair-propagation sentiment signals.
  • Expand the implemented topic features with acceleration and propagation measures.
  • Compare NMF, LDA, embedding clustering, FinBERT-only, and timestamp-shuffled topic controls.
  • Add explicit source-coverage and missingness features.
  • Evaluate text-time shuffling as a permanent placebo.
  • Cache entity linking, event tags, model version, text hash, and FinBERT output permanently.
  • Test tactical triggers only on exact-timestamp data with next-open fills and identical costs.

Related documentation