Redesign CausalPy method selection skill

causalpy/skills/README.md

@@ -10,7 +10,7 @@ When these user-facing skills change, update the distributed Decision AI Hub cop
 
 | Path | Purpose |
 |------|---------|
-| `choosing-causalpy-methods/` | Choosing the right CausalPy experiment class from a causal question and data structure |
+| `choosing-causalpy-methods/` | Route causal or impact questions to the right CausalPy experiment via ordered intake, disambiguation, and explicit no-fit outcomes |
 | `causal-detective/` | Challenging causal claims with threat assessment, counterfactual reasoning, and falsification checks |
 | `example-datasets/` | Loading bundled CausalPy example datasets for demos, tutorials, and tests |
 | `running-causalpy-experiments/` | Fitting chosen experiments, configuring models and priors, summarizing, plotting, and interpreting results |

causalpy/skills/choosing-causalpy-methods/SKILL.md

@@ -1,46 +1,76 @@
 ---
 name: choosing-causalpy-methods
-description: Choose the appropriate CausalPy experiment class from a causal question, data structure, treatment assignment, and identification assumptions. Use before writing analysis code when the method is not yet settled.
+description: Choose the appropriate CausalPy experiment class from a causal or impact question, data structure, treatment assignment, and identification assumptions. Use before writing analysis code when the method is not yet settled, including plain-English questions about whether a campaign, policy, or intervention worked.
 ---
 
 # Choosing CausalPy Methods
 
-Use this skill to translate a user's causal question into a CausalPy experiment choice. This is the design-intake skill, not the implementation skill. Once the method is chosen, hand off to `running-causalpy-experiments` for constructor details, model configuration, priors, summaries, plots, and interpretation.
+Use this skill to translate a user's causal or impact question into a CausalPy experiment choice, including plain-English questions like "did the campaign work?", "what was the effect of the rollout?", or "did the policy change sales?". See [Skill triggers](reference/triggers.md) for additional discovery keywords. This is the design-intake skill, not the implementation skill. Optimize for agent use: follow the ordered routing steps, prefer explicit uncertainty over force-fitting, and do not write analysis code until the method route is matched or the user has answered the key ambiguity. Once the method is chosen, hand off to `running-causalpy-experiments` for constructor details, model configuration, priors, summaries, plots, and interpretation.
 
-## Intake Checklist
+## Required Intake
 
-1. Restate the estimand: ATE, ATT, local threshold effect, treatment-on-treated over time, cumulative impact, or a policy/campaign lift.
-2. Identify the data shape: single time series, wide panel of units, long panel of unit-time rows, cross-section, or pre/post group data.
-3. Identify treatment assignment: known intervention time, staggered adoption, threshold/cutoff, kink, instrument, observed treatment with confounders, or treated unit plus donor pool.
-4. Check the identifying story: parallel trends, no anticipation, no manipulation at cutoff, valid instrument, overlap/positivity, convex hull/donor support, or trend continuity.
-5. Recommend one primary CausalPy experiment and any plausible alternatives, then explain the extra data or assumptions needed to choose among them.
+Before naming a method, identify these facts. If the request is missing several, ask for the single most decision-relevant missing fact.
 
-## Fast Routing
+1. Estimand: ATE, ATT, local threshold effect, event-study path, cumulative post-period impact, baseline-adjusted group contrast, or coefficient-level association.
+2. Assignment mechanism: known intervention time, common pre/post group treatment, staggered adoption, cutoff, kink, instrument, observed treatment with measured confounders, or no credible assignment story.
+3. Data topology: single outcome series, wide unit-by-time panel, long unit-time panel, cross-section, or single pre/post observations by unit or group.
+4. Controls: none, donor units, treated/control groups, never-treated cohorts, measured covariates, near-cutoff observations, or instruments.
+5. Identification support: pre-period history, donor support, overlap/positivity, no anticipation, absorbing treatment, no manipulation at cutoff, instrument validity, or baseline adjustment.
+6. Backend/reporting constraints: whether the user needs OLS/sklearn, Bayesian uncertainty, `effect_summary()`, or a unified `plot()`.
 
-- One treated time series, known intervention time, no donor pool: `InterruptedTimeSeries`.
-- Known level/slope changes in one time series, especially multiple interruptions: `PiecewiseITS`.
-- Treated and control groups observed before and after one intervention: `DifferenceInDifferences`.
-- Units adopt treatment at different times: `StaggeredDifferenceInDifferences`.
-- One or more treated units with multiple untreated donor units in wide panel format: `SyntheticControl`.
-- Synthetic-control setting where both unit weights and pre-period time weights are part of the design: `SyntheticDifferenceInDifferences`.
-- Panel regression or fixed-effects adjustment is the target rather than a named quasi-experimental design: `PanelRegression`.
-- Pretest/posttest nonequivalent groups with a baseline outcome: `PrePostNEGD`.
-- Treatment assigned by crossing a cutoff in a running variable: `RegressionDiscontinuity`.
-- Treatment intensity changes slope at a threshold rather than jumping in level: `RegressionKink`.
-- Treatment is endogenous but there is a credible instrument: `InstrumentalVariable`.
-- Observational binary treatment with measured confounders and overlap: `InversePropensityWeighting`.
+## Routing Workflow
 
-## Output Pattern
+Use the canonical routing algorithm in [Decision tree](reference/decision_tree.md). It is deliberately written as text/pseudocode, not a visual decision tree, so agents can follow it linearly. Do not skip from a keyword such as "time series" or "panel" directly to a class; route through assignment mechanism, data topology, controls, and disqualifiers.
 
-When you use this skill, return:
+When a route is close but not settled, use the disambiguation cards:
 
-- Recommended method: name the CausalPy experiment class.
-- Why it fits: tie the recommendation to data shape, assignment mechanism, and estimand.
-- Required columns/data layout: list the minimal data structure needed.
-- Key assumptions: state what must be credible for a causal interpretation.
-- Main risks: name obvious failure modes or sensitivity checks.
-- Next step: route to `running-causalpy-experiments` and the relevant method reference.
+- [ITS vs Piecewise ITS](reference/disambiguation/its_vs_piecewise_its.md)
+- [ITS vs Synthetic Control vs DiD](reference/disambiguation/its_vs_sc_vs_did.md)
+- [DiD vs Staggered DiD vs Panel Regression](reference/disambiguation/did_vs_staggered_vs_panel.md)
+- [Synthetic Control vs SDiD](reference/disambiguation/sc_vs_sdid.md)
+- [IPW vs IV vs Panel Regression](reference/disambiguation/ipw_vs_iv_vs_panel.md)
+- [PrePostNEGD vs DiD](reference/disambiguation/prepostnegd_vs_did.md)
+
+## Output Contracts
+
+Return exactly one of these outcomes.
+
+### Matched
+
+- Recommended method: name one primary CausalPy experiment class.
+- Why it fits: tie the recommendation to estimand, assignment mechanism, data topology, and controls.
+- Disqualifiers checked: name the main alternatives rejected and why.
+- Required columns/data layout: list the minimal structure needed.
+- Key assumptions: state what must be credible for causal interpretation.
+- Main risks: name likely failure modes and sensitivity checks.
+- Next step: route to `running-causalpy-experiments` and the relevant method reference. If the user wants to stress-test the claim before trusting it, also suggest `causal-detective`.
+
+### Ambiguous
+
+- Candidate methods: name the top two plausible CausalPy classes.
+- What separates them: state the concrete distinction, such as forecast counterfactual vs segmented level/slope model.
+- Deciding question: ask one question that will resolve the route.
+- Next step: do not write analysis code until the user answers.
+
+### Not Identifiable Yet
+
+- Status: the method may exist in CausalPy, but the available data or assumptions are insufficient.
+- Missing requirement: name the specific missing design fact, such as no pre-period, no donor support, no overlap check, or no credible instrument.
+- Deciding question: ask for the missing fact or suggest what evidence would be needed.
+- Next step: do not force a method recommendation.
+
+### Not Implemented In CausalPy
+
+- Status: state "CausalPy has not implemented the right method."
+- Ideal method category: name the method family the user likely needs.
+- Why no CausalPy experiment fits: explain the mismatch with assignment, estimand, treatment type, data topology, or reporting needs.
+- Closest partial fit: mention a CausalPy class only if it is genuinely useful, and state its limitations.
+- What would unlock CausalPy: describe the data or assumption change that would make an implemented experiment appropriate.
 
 ## References
 
+- [Skill triggers](reference/triggers.md)
+- [Decision tree](reference/decision_tree.md)
 - [Experiment decision guide](reference/experiment_decision_guide.md)
+- [Method capability matrix](reference/method_capability_matrix.md)
+- [When CausalPy has not implemented the right method](reference/not_in_causalpy.md)

causalpy/skills/choosing-causalpy-methods/reference/decision_tree.md

@@ -0,0 +1,93 @@
+# Decision Tree
+
+Use this file as the canonical routing algorithm for choosing a CausalPy experiment. It is written as ordered text for agents, not as a visual tree. Follow the steps in order and stop as soon as the route is matched, ambiguous, not identifiable yet, or not implemented in CausalPy.
+
+## Routing Algorithm
+
+```text
+1. Check required intake.
+   If estimand, assignment mechanism, data topology, and control type are not known, ask for the single missing fact that most changes the route.
+
+2. Check for unsupported goals before force-fitting.
+   If the user needs matching, mediation, survival models, causal forests/CATE, fuzzy RD, non-absorbing staggered treatment, augmented synthetic control, matrix completion, alternative staggered DiD estimators, or continuous/multi-arm treatment outside IV or kink settings, return the Not implemented in CausalPy output.
+
+3. If assignment is a known intervention time, route by data topology.
+   If data is one outcome series with no donor units, compare InterruptedTimeSeries and PiecewiseITS.
+   If data is one treated outcome series plus one or more comparison/control series used as formula predictors, route to InterruptedTimeSeries as Comparative Interrupted Time Series (CITS), then compare against SyntheticControl if the user wants constrained donor weighting.
+   If data is a wide panel where columns are units and at least one untreated donor unit is available, compare SyntheticControl and SyntheticDifferenceInDifferences.
+   If data is a long panel with treated and control groups, compare DifferenceInDifferences, StaggeredDifferenceInDifferences, and PanelRegression.
+
+4. If assignment is a common treated/control pre/post design, route to DifferenceInDifferences.
+   Use this when treated and control groups share one intervention timing and the estimand is the group-by-post interaction. If adoption timing differs by unit, switch to StaggeredDifferenceInDifferences. If there is only one baseline and one post outcome per unit, compare PrePostNEGD.
+
+5. If assignment is staggered adoption, route to StaggeredDifferenceInDifferences only when treatment is absorbing.
+   Use this for cohort or event-time ATT paths with unit-time panel data, no anticipation, and never-treated or not-yet-treated comparison information. Confirm each calendar period in the estimand has at least one untreated unit; otherwise return Not identifiable yet. If treatment can turn off or repeat, return Not implemented in CausalPy.
+
+6. If assignment is a cutoff or kink in a running variable, route by estimand.
+   Use RegressionDiscontinuity for a sharp level jump at a cutoff. Use RegressionKink for a slope or treatment-intensity change at a kink point. If treatment probability changes but treatment is not sharp at the cutoff, return Not implemented in CausalPy unless the problem can be reframed as a general InstrumentalVariable analysis with clear caveats.
+
+7. If assignment depends on a credible instrument, route to InstrumentalVariable.
+   Require a relevance story, exclusion restriction, and outcome/treatment/instrument data. If the user mainly has measured confounders rather than an instrument, compare InversePropensityWeighting and PanelRegression instead.
+
+8. If treatment is observed and confounded, route by treatment type and adjustment story.
+   Use InversePropensityWeighting only for binary treatment with measured confounders and plausible overlap. Use PanelRegression only when the analysis target is fixed-effects coefficient estimation and the user understands the result is not automatically a causal design. If confounding is unmeasured and there is no valid instrument, return Not identifiable yet or Not implemented in CausalPy rather than forcing IPW.
+
+9. If data contains pre and post outcomes but not a repeated time panel, compare PrePostNEGD and DifferenceInDifferences.
+   Use PrePostNEGD for baseline-adjusted nonequivalent group designs with one pretest and one posttest outcome. Use DifferenceInDifferences only when there are repeated observations over time that support a pre/post group trend comparison.
+
+10. Apply capability gates before finalizing.
+    If the user requires OLS/sklearn, avoid Bayesian-only methods. If the user requires `effect_summary()` or a unified `plot()`, check the method capability matrix before selecting IV, IPW, or PanelRegression.
+
+11. Return one output contract.
+    Return Matched, Ambiguous, Not identifiable yet, or Not implemented in CausalPy. Do not write analysis code as part of this skill unless the route is Matched and the user explicitly asks to proceed.
+```
+
+## Leaf Routes
+
+### `InterruptedTimeSeries`
+
+Use when there is one outcome series, a known intervention time, and the counterfactual should be forecast from the pre-intervention trend. This also covers Comparative Interrupted Time Series (CITS) when comparison/control series are included as predictors in the formula. Do not use it when the user wants synthetic-control donor weights or has a treated/control panel DiD design.
+
+### `PiecewiseITS`
+
+Use when there is one outcome series and the estimand is an explicit level and/or slope change at one or more known interruption times, encoded through `step()` or `ramp()` formula terms. Do not use it for unknown changepoint discovery.
+
+### `DifferenceInDifferences`
+
+Use for treated and control groups observed before and after a common intervention timing, where the estimand is the treated-group by post-period contrast. Do not use it for staggered adoption or one-row pre/post snapshots without repeated time structure.
+
+### `StaggeredDifferenceInDifferences`
+
+Use for long unit-time panels where units adopt treatment at different times and treatment is absorbing. Requires never-treated or not-yet-treated units so each calendar period has untreated support for time fixed effects. Do not use it for reversible, intermittent, or non-absorbing treatments.
+
+### `SyntheticControl`
+
+Use for wide panels with treated unit columns and untreated donor unit columns, where donor weights should reproduce the treated unit's pre-period path. Do not use it when there is no credible donor support.
+
+### `SyntheticDifferenceInDifferences`
+
+Use for the same wide donor-panel setting as synthetic control when the design explicitly relies on both donor-unit weights and pre-period time weights. Treat it as Bayesian-only in practice.
+
+### `PanelRegression`
+
+Use when fixed-effects regression is the target, especially for coefficient-level adjustment with unit and/or time effects. Do not present it as a causal design unless the treatment assignment story justifies that interpretation.
+
+### `PrePostNEGD`
+
+Use for pretest/posttest nonequivalent group designs with a baseline outcome, post outcome, and group indicator. Do not use it for repeated panel time series where DiD assumptions can be assessed.
+
+### `RegressionDiscontinuity`
+
+Use for sharp regression discontinuity designs where treatment jumps at a known cutoff in a running variable. Do not use it for fuzzy RD unless the user accepts a different IV framing.
+
+### `RegressionKink`
+
+Use when treatment intensity or incentives change slope at a known kink point rather than jumping in level. Do not use it for ordinary threshold discontinuities.
+
+### `InstrumentalVariable`
+
+Use when treatment is endogenous and there is a credible instrument. Do not use it just because a covariate predicts treatment; the exclusion restriction must be part of the design story.
+
+### `InversePropensityWeighting`
+
+Use for observational binary treatment with measured confounders and plausible overlap. Do not use it for continuous, multi-arm, or unmeasured-confounding problems.

causalpy/skills/choosing-causalpy-methods/reference/disambiguation/did_vs_staggered_vs_panel.md

@@ -0,0 +1,30 @@
+# DiD vs Staggered DiD vs Panel Regression
+
+Use this card when the user has panel or repeated group-time data with treatment timing.
+
+## Deciding Question
+
+Is treatment timing common, staggered and absorbing, or not a quasi-experimental timing design at all?
+
+## Choose `DifferenceInDifferences`
+
+- There is one treated group and one control group observed before and after a common intervention timing.
+- The formula can express a single treated-group by post-period interaction.
+- The identifying assumption is parallel trends for treated and control groups.
+
+## Choose `StaggeredDifferenceInDifferences`
+
+- Units adopt treatment at different times.
+- Treatment is absorbing: once treated, a unit stays treated.
+- The estimand is a cohort, event-time, or overall ATT path under no anticipation and parallel trends.
+- Each calendar period in the estimand needs at least one untreated unit. Without never-treated or not-yet-treated units, time fixed effects are not identified and CausalPy warns or fails.
+
+## Choose `PanelRegression`
+
+- The target is coefficient-level fixed-effects regression, not a named DiD design.
+- The user wants unit and/or time fixed effects for adjusted association or for a formula they will defend separately.
+- The route should warn that `PanelRegression` is not a causal design by itself and does not implement `effect_summary()`.
+
+## Choose Neither
+
+- Return Not implemented in CausalPy for non-absorbing staggered treatment, repeated treatment on/off, or a request for a specific alternative staggered estimator such as Callaway-Sant'Anna, Sun-Abraham, or Gardner.

causalpy/skills/choosing-causalpy-methods/reference/disambiguation/ipw_vs_iv_vs_panel.md

@@ -0,0 +1,31 @@
+# IPW vs IV vs Panel Regression
+
+Use this card when treatment is observational or potentially confounded.
+
+## Deciding Question
+
+What is the identification strategy: measured-confounder adjustment with overlap, a valid instrument, or fixed-effects regression?
+
+## Choose `InversePropensityWeighting`
+
+- Treatment is binary.
+- Confounders needed for conditional exchangeability are measured.
+- Positivity and overlap are plausible.
+- The target is a weighted ATE-style contrast, and the user does not need `effect_summary()` or unified `plot()`.
+
+## Choose `InstrumentalVariable`
+
+- Treatment is endogenous or unmeasured confounding is central.
+- There is a credible instrument with relevance, exclusion, and exogeneity arguments.
+- The user accepts Bayesian IV with custom inspection rather than standard `summary()`, `effect_summary()`, and `plot()` outputs.
+
+## Choose `PanelRegression`
+
+- The data is a panel and the target is fixed-effects coefficient estimation.
+- Unit and/or time effects are part of the adjustment strategy.
+- The answer should be framed as coefficient-level regression unless the treatment assignment story supports causal interpretation.
+
+## Choose Neither
+
+- Return Not identifiable yet if confounders are unmeasured and no credible instrument exists.
+- Return Not implemented in CausalPy for matching, multi-arm treatment, continuous dose-response treatment, causal forests, or mediation.