Skip to content

Commit c107d4e

Browse files
Improve user/agent facing skills (#929)
* Clarify user-facing CausalPy skills Rename the method-selection and execution skills so agents can route causal-design work separately from experiment-running guidance, and expand the execution references with current experiment coverage and scale-aware prior guidance. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix agent-facing CausalPy skill examples Clarify unsupported experiment outputs and make the expanded execution references less likely to send agents into runtime errors. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent d5d1fe0 commit c107d4e

26 files changed

Lines changed: 980 additions & 281 deletions
Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
# Causal Estimators
22

3-
This command points to the `performing-causal-analysis` Skill in
4-
`causalpy/skills/performing-causal-analysis/` for estimation, summaries, and plots.
3+
This command points to the `running-causalpy-experiments` Skill in
4+
`causalpy/skills/running-causalpy-experiments/` for estimation, summaries, priors, and plots.

.cursor/commands/causalpy_methods.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,5 @@
22

33
This command points to Skills in `causalpy/skills/`:
44

5-
- `designing-experiments` for method selection
6-
- `performing-causal-analysis` for method usage and references
5+
- `choosing-causalpy-methods` for method selection
6+
- `running-causalpy-experiments` for method usage and references
Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
# Causal Research
22

3-
This command points to the `designing-experiments` Skill in
4-
`causalpy/skills/designing-experiments/` for method selection guidance.
3+
This command points to the `choosing-causalpy-methods` Skill in
4+
`causalpy/skills/choosing-causalpy-methods/` for method selection guidance.

causalpy/skills/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,6 @@ Developer-focused skills (environment setup, PR workflows, testing conventions,
88

99
| Path | Purpose |
1010
|------|---------|
11-
| `designing-experiments/` | Choosing the right quasi-experimental method |
12-
| `performing-causal-analysis/` | Fitting models, estimating impacts, plotting results |
11+
| `choosing-causalpy-methods/` | Choosing the right CausalPy experiment class from a causal question and data structure |
12+
| `running-causalpy-experiments/` | Fitting chosen experiments, configuring models and priors, summarizing, plotting, and interpreting results |
1313
| `running-placebo-analysis/` | Placebo-in-time sensitivity checks |
Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
---
2+
name: choosing-causalpy-methods
3+
description: Choose the appropriate CausalPy experiment class from a causal question, data structure, treatment assignment, and identification assumptions. Use before writing analysis code when the method is not yet settled.
4+
---
5+
6+
# Choosing CausalPy Methods
7+
8+
Use this skill to translate a user's causal question into a CausalPy experiment choice. This is the design-intake skill, not the implementation skill. Once the method is chosen, hand off to `running-causalpy-experiments` for constructor details, model configuration, priors, summaries, plots, and interpretation.
9+
10+
## Intake Checklist
11+
12+
1. Restate the estimand: ATE, ATT, local threshold effect, treatment-on-treated over time, cumulative impact, or a policy/campaign lift.
13+
2. Identify the data shape: single time series, wide panel of units, long panel of unit-time rows, cross-section, or pre/post group data.
14+
3. Identify treatment assignment: known intervention time, staggered adoption, threshold/cutoff, kink, instrument, observed treatment with confounders, or treated unit plus donor pool.
15+
4. Check the identifying story: parallel trends, no anticipation, no manipulation at cutoff, valid instrument, overlap/positivity, convex hull/donor support, or trend continuity.
16+
5. Recommend one primary CausalPy experiment and any plausible alternatives, then explain the extra data or assumptions needed to choose among them.
17+
18+
## Fast Routing
19+
20+
- One treated time series, known intervention time, no donor pool: `InterruptedTimeSeries`.
21+
- Known level/slope changes in one time series, especially multiple interruptions: `PiecewiseITS`.
22+
- Treated and control groups observed before and after one intervention: `DifferenceInDifferences`.
23+
- Units adopt treatment at different times: `StaggeredDifferenceInDifferences`.
24+
- One or more treated units with multiple untreated donor units in wide panel format: `SyntheticControl`.
25+
- Synthetic-control setting where both unit weights and pre-period time weights are part of the design: `SyntheticDifferenceInDifferences`.
26+
- Panel regression or fixed-effects adjustment is the target rather than a named quasi-experimental design: `PanelRegression`.
27+
- Pretest/posttest nonequivalent groups with a baseline outcome: `PrePostNEGD`.
28+
- Treatment assigned by crossing a cutoff in a running variable: `RegressionDiscontinuity`.
29+
- Treatment intensity changes slope at a threshold rather than jumping in level: `RegressionKink`.
30+
- Treatment is endogenous but there is a credible instrument: `InstrumentalVariable`.
31+
- Observational binary treatment with measured confounders and overlap: `InversePropensityWeighting`.
32+
33+
## Output Pattern
34+
35+
When you use this skill, return:
36+
37+
- Recommended method: name the CausalPy experiment class.
38+
- Why it fits: tie the recommendation to data shape, assignment mechanism, and estimand.
39+
- Required columns/data layout: list the minimal data structure needed.
40+
- Key assumptions: state what must be credible for a causal interpretation.
41+
- Main risks: name obvious failure modes or sensitivity checks.
42+
- Next step: route to `running-causalpy-experiments` and the relevant method reference.
43+
44+
## References
45+
46+
- [Experiment decision guide](reference/experiment_decision_guide.md)
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Experiment Decision Guide
2+
3+
Use this guide when an agent needs to choose a CausalPy experiment before writing analysis code. Prefer the simplest design that matches the data-generating story and has defensible identification assumptions.
4+
5+
## Time-Series Designs
6+
7+
| Situation | CausalPy class | Data shape | Main assumption | Common alternative |
8+
|---|---|---|---|---|
9+
| One outcome series, one intervention time, forecast counterfactual from pre-period | `InterruptedTimeSeries` | DataFrame indexed by time or with a time column | Pre-intervention trend model remains valid absent treatment | `PiecewiseITS` when estimating explicit level/slope changes in the full series |
10+
| Known level or slope changes, possibly multiple interruptions | `PiecewiseITS` | Single time series with time variable used in `step()` / `ramp()` formula terms | Functional form captures untreated trend plus intervention changes | `InterruptedTimeSeries` when the goal is pre-period forecasting |
11+
| Treated series plus donor units measured over time | `SyntheticControl` | Wide panel, columns are units | Donor pool can reproduce treated pre-period and supports counterfactual | `SyntheticDifferenceInDifferences` when time weights are part of the design |
12+
| Synthetic-control setting with unit weights and pre-period time weights | `SyntheticDifferenceInDifferences` | Wide panel, columns are units | Weighted controls and weighted pre-periods provide a credible counterfactual | `SyntheticControl` for simpler donor-weight-only analysis |
13+
14+
## Panel And Group Designs
15+
16+
| Situation | CausalPy class | Data shape | Main assumption | Common alternative |
17+
|---|---|---|---|---|
18+
| One treated group and one control group before/after treatment | `DifferenceInDifferences` | Long panel or repeated group-time observations with treatment and post indicators | Parallel trends between treated and control groups | `StaggeredDifferenceInDifferences` for staggered adoption |
19+
| Units adopt treatment at different times | `StaggeredDifferenceInDifferences` | Long unit-time panel with treatment timing | Parallel trends, no anticipation, absorbing treatment | `PanelRegression` for fixed-effects association or simpler adjustment |
20+
| Fixed-effects regression is the analysis target | `PanelRegression` | Long panel with unit identifiers and optional time identifiers | Fixed effects control relevant unit/time confounding | DiD variants when treatment timing is central |
21+
| Treated/control groups with baseline and post outcome | `PrePostNEGD` | Cross-sectional or group data with pre and post outcomes | Baseline outcome adjustment captures pre-existing group differences | `DifferenceInDifferences` when repeated observations over time are available |
22+
23+
## Threshold And Cross-Sectional Designs
24+
25+
| Situation | CausalPy class | Data shape | Main assumption | Common alternative |
26+
|---|---|---|---|---|
27+
| Treatment switches at a cutoff in a running variable | `RegressionDiscontinuity` | Cross-section or repeated observations with running variable and threshold | Units near cutoff are comparable and cannot precisely manipulate assignment | `RegressionKink` when slope changes rather than treatment level |
28+
| Treatment intensity changes slope at a threshold | `RegressionKink` | Cross-section or repeated observations with running variable and kink point | Smooth potential outcomes around the kink absent treatment-intensity change | `RegressionDiscontinuity` for jump discontinuities |
29+
| Endogenous treatment with valid instrument | `InstrumentalVariable` | DataFrame for outcome/treatment covariates plus instruments | Instrument relevance, exclusion, and no unblocked instrument-outcome path | `InversePropensityWeighting` if treatment is confounded but not instrumented |
30+
| Observed binary treatment with measured confounders | `InversePropensityWeighting` | Cross-section or panel rows with treatment, outcome, and confounders | Conditional exchangeability and overlap/positivity | Outcome regression or doubly robust workflows when available |
31+
32+
## Method Selection Questions
33+
34+
- Is treatment assigned by time, by unit, by threshold, by instrument, or by observed covariates?
35+
- Is the estimand a total post-period impact, an event-study path, a local threshold effect, or an adjusted treatment contrast?
36+
- Are controls actual untreated units, donor time series, never-treated cohorts, or measured covariates?
37+
- Does the data have enough pre-treatment history to support trend, donor, or placebo checks?
38+
- Is the assignment mechanism credible enough for causal language, or should the agent frame the result as descriptive?
39+
40+
## Handoff To Execution
41+
42+
After selecting a method, use `running-causalpy-experiments` for the concrete workflow: data preparation, constructor arguments, model choice, scale-aware priors, summaries, plots, effect summaries, and sensitivity checks.

causalpy/skills/designing-experiments/SKILL.md

Lines changed: 0 additions & 31 deletions
This file was deleted.

causalpy/skills/performing-causal-analysis/SKILL.md

Lines changed: 0 additions & 29 deletions
This file was deleted.

causalpy/skills/performing-causal-analysis/reference/diff_in_diff.md

Lines changed: 0 additions & 52 deletions
This file was deleted.

causalpy/skills/performing-causal-analysis/reference/interrupted_time_series.md

Lines changed: 0 additions & 53 deletions
This file was deleted.

0 commit comments

Comments
 (0)