Skip to content

Add a pymc_forecast model-provider adapter (ITS/SC counterfactual backend) #1013

Description

@daimon-pymclabs

Summary

Add a thin adapter that lets a pymc_forecast forecasting model act as a model provider behind CausalPy's existing experiment API, in the same spirit as the scikit-learn wrapper (causalpy/skl_models.py + experiments/model_adapter.py). CausalPy keeps identification, counterfactual construction, and placebo methods; pymc_forecast provides the fitted forecasting model.

This is the outcome of a design discussion with the pymc_forecast maintainers. The five upstream API changes CausalPy needs are implemented, reviewed, and shipped (see below).

Package naming: the PyPI distribution is pymc-forecast (hyphen); the import name is pymc_forecast (underscore). Install with pip install pymc-forecast>=0.1.0.

Scope — first pass is ITS-only

In (this issue): pymc_forecast as a selectable time-series model backend for InterruptedTimeSeries.

Deferred (follow-up): SyntheticControl as a second backend target. Left out of the first pass to keep the adapter minimal and prove the boundary on one experiment before generalizing. Tracked as a follow-up once ITS lands.

Out (both passes): no outsourcing of placebo-in-time, power analysis, or power curves. Placebo-in-time is a cross-method quasi-experimental primitive and stays in CausalPy; back-tests/cross-validation are not causal tests and must not be reframed as such. Keep the boundary at model-provider + forecasting metrics.

Adapter mapping

The ITS experiment (experiments/interrupted_time_series.py) drives a backend through fit(X, y, coords), predict(X) (in-sample, pre-period), predict(X, out_of_sample=True) (the counterfactual, post-period), and score(X, y). Mapping onto pymc_forecast:

  • fit(X, y) → construct + fit the forecasting model on the pre-period.
  • predict(X) (pre) → predict_in_sample(...); take the posterior_predictive group.
  • predict(X, out_of_sample=True) (post, "as if untreated") → .forecast(future_covariates=post_X) when the model has exogenous regressors, or .forecast(future_index=post_index) for a covariate-free trend/seasonal counterfactual.
  • Extract draw-level samples via prediction_samples(result), then rename the documented dims onto CausalPy coords: time_futureobs_ind, seriestreated_units.
  • Priors flow through the model object (pymc-extras Prior), so users keep the transparent-prior ethos.

Upstream dependencies (pymc-labs/pymc_forecast) — shipped in 0.1.0

Tasks (ITS-only first pass)

  • Upstream API shipped in a release: pymc-forecast==0.1.0.
  • Add pymc-forecast>=0.1.0 as an optional extra (not a hard dep), mirroring the pymc-marketing extra.
  • Implement the adapter (target: comparable in size to the sklearn wrapper).
  • Wire it as a selectable backend for InterruptedTimeSeries; no change to the public API/UX ethos.
  • Round-trip test: fit pre / forecast post-as-untreated / calculate_impact on draw-level samples, checked against the existing pymc BSTS path.
  • Docs note on when to reach for this backend vs the existing PyMCModel classes (ties into Improve agent discoverability of user-facing skills, and review graded-treatment routing in choosing-causalpy-methods #992 method-routing).
  • Follow-up (separate issue): extend the adapter to SyntheticControl once ITS is proven.

Caveats

  • pymc-forecast is young (0.1.0); introduce behind an optional extra and a version pin, mirroring how we already manage pymc/pytensor pins. Watch for API movement across 0.x minors — the output schema is the one surface upstream has committed to keeping stable (contract-tested, changelog-gated).
  • The interval-calibration-under-input-scaling concern (raised externally re: synthetic control) stays a CausalPy-side responsibility; the upstream prior injectability is the mechanism to plumb it, not a fix.

Related

Feeds/relates to #982 (graduate a BSTS default), #980 (exogenous regressors on StateSpaceTimeSeries), and the temporal-validation idea in #367.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions