|
| 1 | +# Experiment Decision Guide |
| 2 | + |
| 3 | +Use this guide when an agent needs to choose a CausalPy experiment before writing analysis code. Prefer the simplest design that matches the data-generating story and has defensible identification assumptions. |
| 4 | + |
| 5 | +## Time-Series Designs |
| 6 | + |
| 7 | +| Situation | CausalPy class | Data shape | Main assumption | Common alternative | |
| 8 | +|---|---|---|---|---| |
| 9 | +| One outcome series, one intervention time, forecast counterfactual from pre-period | `InterruptedTimeSeries` | DataFrame indexed by time or with a time column | Pre-intervention trend model remains valid absent treatment | `PiecewiseITS` when estimating explicit level/slope changes in the full series | |
| 10 | +| Known level or slope changes, possibly multiple interruptions | `PiecewiseITS` | Single time series with time variable used in `step()` / `ramp()` formula terms | Functional form captures untreated trend plus intervention changes | `InterruptedTimeSeries` when the goal is pre-period forecasting | |
| 11 | +| Treated series plus donor units measured over time | `SyntheticControl` | Wide panel, columns are units | Donor pool can reproduce treated pre-period and supports counterfactual | `SyntheticDifferenceInDifferences` when time weights are part of the design | |
| 12 | +| Synthetic-control setting with unit weights and pre-period time weights | `SyntheticDifferenceInDifferences` | Wide panel, columns are units | Weighted controls and weighted pre-periods provide a credible counterfactual | `SyntheticControl` for simpler donor-weight-only analysis | |
| 13 | + |
| 14 | +## Panel And Group Designs |
| 15 | + |
| 16 | +| Situation | CausalPy class | Data shape | Main assumption | Common alternative | |
| 17 | +|---|---|---|---|---| |
| 18 | +| One treated group and one control group before/after treatment | `DifferenceInDifferences` | Long panel or repeated group-time observations with treatment and post indicators | Parallel trends between treated and control groups | `StaggeredDifferenceInDifferences` for staggered adoption | |
| 19 | +| Units adopt treatment at different times | `StaggeredDifferenceInDifferences` | Long unit-time panel with treatment timing | Parallel trends, no anticipation, absorbing treatment | `PanelRegression` for fixed-effects association or simpler adjustment | |
| 20 | +| Fixed-effects regression is the analysis target | `PanelRegression` | Long panel with unit identifiers and optional time identifiers | Fixed effects control relevant unit/time confounding | DiD variants when treatment timing is central | |
| 21 | +| Treated/control groups with baseline and post outcome | `PrePostNEGD` | Cross-sectional or group data with pre and post outcomes | Baseline outcome adjustment captures pre-existing group differences | `DifferenceInDifferences` when repeated observations over time are available | |
| 22 | + |
| 23 | +## Threshold And Cross-Sectional Designs |
| 24 | + |
| 25 | +| Situation | CausalPy class | Data shape | Main assumption | Common alternative | |
| 26 | +|---|---|---|---|---| |
| 27 | +| Treatment switches at a cutoff in a running variable | `RegressionDiscontinuity` | Cross-section or repeated observations with running variable and threshold | Units near cutoff are comparable and cannot precisely manipulate assignment | `RegressionKink` when slope changes rather than treatment level | |
| 28 | +| Treatment intensity changes slope at a threshold | `RegressionKink` | Cross-section or repeated observations with running variable and kink point | Smooth potential outcomes around the kink absent treatment-intensity change | `RegressionDiscontinuity` for jump discontinuities | |
| 29 | +| Endogenous treatment with valid instrument | `InstrumentalVariable` | DataFrame for outcome/treatment covariates plus instruments | Instrument relevance, exclusion, and no unblocked instrument-outcome path | `InversePropensityWeighting` if treatment is confounded but not instrumented | |
| 30 | +| Observed binary treatment with measured confounders | `InversePropensityWeighting` | Cross-section or panel rows with treatment, outcome, and confounders | Conditional exchangeability and overlap/positivity | Outcome regression or doubly robust workflows when available | |
| 31 | + |
| 32 | +## Method Selection Questions |
| 33 | + |
| 34 | +- Is treatment assigned by time, by unit, by threshold, by instrument, or by observed covariates? |
| 35 | +- Is the estimand a total post-period impact, an event-study path, a local threshold effect, or an adjusted treatment contrast? |
| 36 | +- Are controls actual untreated units, donor time series, never-treated cohorts, or measured covariates? |
| 37 | +- Does the data have enough pre-treatment history to support trend, donor, or placebo checks? |
| 38 | +- Is the assignment mechanism credible enough for causal language, or should the agent frame the result as descriptive? |
| 39 | + |
| 40 | +## Handoff To Execution |
| 41 | + |
| 42 | +After selecting a method, use `running-causalpy-experiments` for the concrete workflow: data preparation, constructor arguments, model choice, scale-aware priors, summaries, plots, effect summaries, and sensitivity checks. |
0 commit comments