Skip to content

feat: add omni-scient-0.1.0 champion-challenger ensemble#167

Open
suryanshsrivastava wants to merge 6 commits into
autogluon:mainfrom
suryanshsrivastava:main
Open

feat: add omni-scient-0.1.0 champion-challenger ensemble#167
suryanshsrivastava wants to merge 6 commits into
autogluon:mainfrom
suryanshsrivastava:main

Conversation

@suryanshsrivastava

Copy link
Copy Markdown

Omni-Scient v0.1.0 — Champion-Challenger Ensemble

A deterministic, pure-stdlib forecasting ensemble. Part of the Omni model family.

Architecture

  • Segmentation: Classifies each series as STABLE, SEASONAL, LUMPY, VOLATILE, or NEW
  • Candidates: SNAIVE (seasonal naive), AVG3 (trailing average), SIDX (seasonal index × trend), Holt-Winters additive
  • Champion selection: Per-series WAPE on a held-out validation window
  • Forecast: Champion retrained on full history

Characteristics

  • Zero pretrained dependencies — trains from scratch on each task
  • Pure stdlib — only numpy beyond the Python standard library
  • Deterministic — same input always produces same output
  • Includes quantile estimates via residual std dev scaling
  • Graceful NaN/inf handling

Files

  • models/omni-scient/model.py — fev model wrapper
  • models/omni-scient/requirements.txt — pinned deps
  • benchmarks/fev_bench/results/omni-scient-0.1.0.csv — full benchmark (100/100 tasks)

Vision

omni-scient is being developed as a foundational time series forecasting model.
This v0.1 is a narrow champion-challenger baseline — the first step toward a pretrained
foundation model in the Omni family.

suryanshsrivastava added 6 commits July 9, 2026 19:54
GM Modular — deterministic, pure-stdlib champion-challenger forecaster.
Segments each series (stable/seasonal/lumpy/volatile/new), runs four
candidate models (SNAIVE, AVG3, SIDX, Holt-Winters), picks champion
per series by WAPE on a held-out validation window.

- Zero pretrained dependencies — trains from scratch on each task
- Includes quantile estimates via residual std dev scaling
- Handles NaN/inf in input data gracefully
@shchur

shchur commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Hi @suryanshsrivastava thanks for your contribution. The scores for the proposed model look much worse than currently the worst baseline considered in the benchmark. I don't think we want to include the proposed model to the leaderboard at this point.

model_name win_rate skill_score
Chronos-2 84.6 47.3
TiRex-2 81.6 45.5
Toto-2.0-2.5B 81.1 44.4
Toto-2.0-1B 80.4 44.4
Toto-2.0-313m 78.6 44.2
TiRex 74.6 42.6
Toto-2.0-22m 72.5 42.9
TimesFM-2.5 71.1 42.2
TabPFN-TS-3 66.9 43.1
FlowState 65.2 39.9
Toto-1.0 62.8 40.8
Toto-2.0-4m 61.8 40.7
TabPFN-TS 61.7 41.5
Moirai-2.0 58 39.3
Chronos-Bolt 57.5 38.9
TFT 43.9 32.2
Sundial-Base 40.1 33.4
Stat. Ensemble 39.2 20.7
PatchTST 38.5 30.1
DeepAR 37.7 29.1
AutoARIMA 34.7 21.3
CatBoost 29.3 23
AutoETS 29.2 -26.8
LightGBM 27.5 21
AutoTheta 24.5 5.5
Seasonal Naive 17.2 0
Naive 11.3 -45.4
Drift 9.9 -45.8
omni-scient-0.1.0 8.4 -91.7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants