Skip to content

feat: add T0Model foundation model#3142

Open
GeoffNN wants to merge 11 commits into
unit8co:masterfrom
theforecastingcompany:feature/add-t0-foundation-model
Open

feat: add T0Model foundation model#3142
GeoffNN wants to merge 11 commits into
unit8co:masterfrom
theforecastingcompany:feature/add-t0-foundation-model

Conversation

@GeoffNN

@GeoffNN GeoffNN commented Jun 23, 2026

Copy link
Copy Markdown

Hi folks! Great work on darts! 🙏
We just released our first open-source model: t0-alpha. Adding it to your library so people can easily use it, benchmark it etc.

Checklist before merging this PR:

  • Mentioned all issues that this PR fixes or addresses.
  • Summarized the updates of this PR under Summary.
  • Added an entry under Unreleased in the Changelog.

Summary

Adds T0Model, a Darts wrapper around The Forecasting Company's open-weights T0 foundation model (~100M-parameter patch transformer for zero-shot forecasting). It follows the existing foundation-model pattern, closest to TiRexModel (delegates to an optional pip package) while also supporting future covariates like Chronos2Model.

  • Subclasses FoundationModel; zero-shot inference for univariate, multivariate, and multiple series.
  • Delegates all forecasting logic and weight loading to the optional tfc-t0 package (from t0 import T0Forecaster), declared in the optional dependency group (not pulled by darts[torch]).
  • Future covariates supported (supports_future_covariates=True): forward() reassembles Darts' (x_past, x_future) into T0's [B, F, context + horizon] covariate layout, NaN-filling the output_chunk_shift gap (T0 treats NaN as missing). Past covariates are not supported.
  • Probabilistic forecasts via QuantileRegression. Any quantiles in (0, 1) are accepted — T0 interpolates levels it was not trained on (it was trained on [0.1, 0.25, 0.5, 0.75, 0.9]).
  • Fine-tuning is explicitly rejected — T0 is a zero-shot inference model (predict() runs under torch.inference_mode()).

Registered in the models lazy-import table, conftest availability flag (T0_AVAILABLE), README / INSTALL / docs model tables, pyproject.toml optional group, and CHANGELOG.md.

Other Information

Tests in darts/tests/models/forecasting/test_t0.py mock T0Forecaster.from_pretrained so CI never downloads weights (gated behind T0_AVAILABLE, matching the TiRex test pattern). Covers creation/validation, deterministic + probabilistic, multivariate, future covariates, and multiple series.

Verified locally against the real theforecastingcompany/t0-alpha weights — univariate deterministic, probabilistic with interpolated quantiles (0.05/0.95), future-covariate, and multivariate forecasts all produce finite outputs of the expected shape.

🤖 Generated with Claude Code

Add T0Model, a Darts wrapper around The Forecasting Company's open-weights
T0 foundation model (theforecastingcompany/t0-alpha on HuggingFace),
following the existing foundation-model pattern (TiRex/Chronos-2).

- Wraps the optional `tfc-t0` package; subclasses `FoundationModel` for
  zero-shot inference (univariate, multivariate, multiple series).
- Supports future covariates (mapped to T0's [B, F, context+horizon] format)
  and QuantileRegression probabilistic forecasts (any quantiles in (0,1);
  T0 interpolates levels it was not trained on).
- Registers in models lazy-import table, conftest availability flag,
  README/INSTALL/docs tables, pyproject `optional` group, and CHANGELOG.
- Tests in test_t0.py mock `T0Forecaster.from_pretrained` (no weight download).
@GeoffNN GeoffNN requested a review from dennisbader as a code owner June 23, 2026 20:47
@daidahao

Copy link
Copy Markdown
Contributor

Nice PR! @GeoffNN

Leaving my two cents here as the original contributor of FoundationModel:

Would you consider relaxing t0 python bounds to 3.10+ so all users can access your model?

Also, is there a reason why past covariates cannot be supported? I have not looked into the model architecture, but would implanting past covariates as part of the context help improve the forecast quality? If so, I would suggest appending past covariates to the context and dropping them from the output.

PS. It would be nice if a technical report / blog post can be provided to help us and users understand the model better.

@GeoffNN

GeoffNN commented Jun 25, 2026

Copy link
Copy Markdown
Author

Hi @daidahao !
Thanks for reviewing 🙏
We treat past covariates as more targets essentially - since the targets are handled in a multi-variate way. I'll fix this in the PR.

The tech report & blog post are both incoming, we'll release this a bit later.
Cheers!

GeoffNN added 2 commits June 25, 2026 15:36
tfc-t0 requires python>=3.11,<3.14 but darts supports python>=3.10, so an
unconditional entry made uv unable to resolve dev-all across darts' full
range, breaking all CI jobs at the sync step. Mirror the onnxruntime pattern
and add a python_version marker.
T0 is variate-agnostic, so past covariates are appended to the context and
forecast jointly with the target, then dropped from the output (per review
feedback from @daidahao). Future covariates continue to use T0's covariate
branch. Flip supports_past_covariates to True; update docstrings, model
tables, changelog, and tests (parametrized over future/past/both).
@dennisbader

Copy link
Copy Markdown
Collaborator

Thanks a lot for this PR @GeoffNN 🚀 Before giving a full review, there are two points:

  • As @daidahao mentioned, can you enable python 3.10 support in the tfc-t0 package itself?
  • All foundation models in Darts currently support (simple) fine-tuning. The idea was that we only add models that support this (since we're open-source). Would it be possible to relax the inference-mode requirement so that the parameters can be updated when running the model? We had a similar conversation on the TiRex integration (here), where the authors also enabled support for it :)

@codecov

codecov Bot commented Jun 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.89%. Comparing base (4ec56a0) to head (1533fbd).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3142      +/-   ##
==========================================
- Coverage   96.94%   96.89%   -0.06%     
==========================================
  Files         164      165       +1     
  Lines       17709    17757      +48     
==========================================
+ Hits        17168    17205      +37     
- Misses        541      552      +11     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@GeoffNN

GeoffNN commented Jun 27, 2026

Copy link
Copy Markdown
Author

Hi both!

  • We're working on enabling the 3.10 requirement. cc @jfainberg on our side
  • Re: finetuning, is it just adding the enable_finetuning flag? From the tirex conversation, I understand that you're not asking for model-specific finetuning code.

Since the model is Apache 2.0, there's no restriction on our side for this!

@dennisbader

Copy link
Copy Markdown
Collaborator

Thanks a lot for the updates @GeoffNN, that sounds great 🚀

Regarding fine-tuning:

  • Yes, it's about the enable_finetuning flag and we don't need model-specific fine-tuning code. Adding the model to our existing foundation model fine-tuning tests here should be enough.
  • From your PR description I read that your model is currently running under the @inference_mode decorator. To enable fine-tuning (parameter updates), you would need to expose a method that is not running under inference mode.

Let me know if it's clear or whether we can support in any way.

@dennisbader

Copy link
Copy Markdown
Collaborator

@GeoffNN another question regarding adding tfc-t0 as a dependency to Darts: I quickly checked out your dependency requirements and noticed that all have an upper version cap here. Is there a specific reason you limit the upper versions? That would mean that at some point, tfc-t0 would block Darts users from installing newer dependency versions. Would it be possible to relax (remove) these caps?

GeoffNN and others added 5 commits July 1, 2026 18:23
- raise_log() no longer takes a logger arg (auto-resolved from caller module).
- PLModuleInput is now a 4-tuple; unpack x_past, x_future, _, _.
…r py3.10 support)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@GeoffNN

GeoffNN commented Jul 11, 2026

Copy link
Copy Markdown
Author

Hello! Sorry for the delay here, we've been traveling - and want to think about the API surface we expose, especially for the with-gradients side.

Re: versions - I believe we gate against untested (future) versions, to avoid breaking the package silently if the dependencies change their API on a major version. Let me double check.

@dennisbader

Copy link
Copy Markdown
Collaborator

Thanks for the updates @GeoffNN 🚀

In the open-source world the continuous dependency integration (e.g. checking against newest versions and then fixing the package continuously if there are some breaking changes in dependencies) is a pro-active way to keep the package up-to-date. So, I would always recommend to go that way (except maybe where you're developing apps or similar where you want to ensure everyone user gets the exact same fixed environment (via dependency lockfiles for example).

Let me know what you think

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants