Summary
Two related gaps surfaced while an AI agent was authoring a new example notebook and had to choose a CausalPy method. Both are about the skills in causalpy/skills/ doing their job at the moment an agent needs them.
- Discovery. User-facing skills live in
causalpy/skills/ (shipped to end users via pip / Decision AI Hub). Agents that rely on auto-discovery only scan .agents/skills/ and its tool-specific twins, so they never see causalpy/skills/. The only bridge is a prose "Skills Location" note partway down AGENTS.md, which is easy to skip.
- Content.
choosing-causalpy-methods appears to route "continuous / graded / dose-response treatment" to not a fit (reference/not_in_causalpy.md), but a graded intervention can often be represented in Piecewise ITS via step() / ramp() terms. The no-fit rule looks too strict and can bounce work out of CausalPy that it can actually handle.
Motivating story
An agent set out to build an example around a graded, event-style treatment. It never found choosing-causalpy-methods, reasoned from general knowledge instead, concluded CausalPy had no quasi-experimental fit for a graded treatment, and nearly redirected the whole example to a different library. A maintainer intervened and pointed at Piecewise ITS, which is exactly the right tool and is described in the shipped skill. So the correct answer was in the repo the entire time; the agent just never reached it, and the one heuristic it might have leaned on would have pushed it the wrong way. Both failure modes are cheap to fix and will recur for any agent (or contributor) doing method selection.
Recommendation 1 — discoverability (keep files where they are)
The split is correct and should stay: user skills ship to end users; developer skills are for maintainers. This is a discovery problem, not a location problem.
- Add a loud, imperative pointer near the top of
AGENTS.md (and CLAUDE.md if present): "Before answering any causal-inference or method-choice question, or authoring an example/notebook, read causalpy/skills/choosing-causalpy-methods/SKILL.md first." Directives get acted on; mid-file location notes get skimmed. Fold in the trigger keywords from the skill's own triggers.md.
- Reframe the dev/user divide explicitly as distribution, not discovery: authoring examples and docs is a maintainer task that requires the user-facing method skills, so dev agents must consult them.
- Optional cheapest auto-discovery bridge without moving or duplicating content: commit a one-line stub at
.agents/skills/choosing-causalpy-methods/SKILL.md whose body just says the real skill lives at causalpy/skills/choosing-causalpy-methods/ and must be read. Shows up in auto-discovery, avoids committed symlinks.
Recommendation 2 — review graded-treatment routing
- Revisit
not_in_causalpy.md and the routing/disambiguation cards so a graded or continuously-dosed intervention at known times routes to Piecewise ITS (step() / ramp() terms, level + slope + cumulative effects) where the design supports it, rather than being classified as out of scope.
- Clarify the genuine boundary: whole-domain dose-response curve estimation vs. graded interventions expressible as segmented terms. The former may be out of scope; the latter is not.
Recommendation 3 — route to pathmc when it is the better fit
When a causal question is genuinely outside CausalPy's quasi-experimental scope but is well suited to a structural-causal-model / DAG approach (e.g. pathmc), the skill should say so and point the user there, rather than only returning "not implemented in CausalPy". A no-fit outcome is far more useful when it names the right neighbouring tool. Keep this at the level of intent: the skill should recognise when a sibling library is the better home and route to it. Exact wording and the boundary between the two are for the skill authors to define.
Suggested acceptance criteria
- An agent working from a fresh checkout is reliably routed to
choosing-causalpy-methods before selecting a method.
choosing-causalpy-methods gives a correct route for a graded intervention at known times (Piecewise ITS), with the no-fit boundary re-scoped to genuine dose-response-curve cases.- No-fit outcomes name a suitable neighbouring tool (such as
pathmc) when the problem is better served there, instead of ending at a bare "not implemented".
Summary
Two related gaps surfaced while an AI agent was authoring a new example notebook and had to choose a CausalPy method. Both are about the skills in
causalpy/skills/doing their job at the moment an agent needs them.causalpy/skills/(shipped to end users via pip / Decision AI Hub). Agents that rely on auto-discovery only scan.agents/skills/and its tool-specific twins, so they never seecausalpy/skills/. The only bridge is a prose "Skills Location" note partway downAGENTS.md, which is easy to skip.choosing-causalpy-methodsappears to route "continuous / graded / dose-response treatment" to not a fit (reference/not_in_causalpy.md), but a graded intervention can often be represented in Piecewise ITS viastep()/ramp()terms. The no-fit rule looks too strict and can bounce work out of CausalPy that it can actually handle.Motivating story
An agent set out to build an example around a graded, event-style treatment. It never found
choosing-causalpy-methods, reasoned from general knowledge instead, concluded CausalPy had no quasi-experimental fit for a graded treatment, and nearly redirected the whole example to a different library. A maintainer intervened and pointed at Piecewise ITS, which is exactly the right tool and is described in the shipped skill. So the correct answer was in the repo the entire time; the agent just never reached it, and the one heuristic it might have leaned on would have pushed it the wrong way. Both failure modes are cheap to fix and will recur for any agent (or contributor) doing method selection.Recommendation 1 — discoverability (keep files where they are)
The split is correct and should stay: user skills ship to end users; developer skills are for maintainers. This is a discovery problem, not a location problem.
AGENTS.md(andCLAUDE.mdif present): "Before answering any causal-inference or method-choice question, or authoring an example/notebook, readcausalpy/skills/choosing-causalpy-methods/SKILL.mdfirst." Directives get acted on; mid-file location notes get skimmed. Fold in the trigger keywords from the skill's owntriggers.md..agents/skills/choosing-causalpy-methods/SKILL.mdwhose body just says the real skill lives atcausalpy/skills/choosing-causalpy-methods/and must be read. Shows up in auto-discovery, avoids committed symlinks.Recommendation 2 — review graded-treatment routing
not_in_causalpy.mdand the routing/disambiguation cards so a graded or continuously-dosed intervention at known times routes to Piecewise ITS (step()/ramp()terms, level + slope + cumulative effects) where the design supports it, rather than being classified as out of scope.Recommendation 3 — route to pathmc when it is the better fit
When a causal question is genuinely outside CausalPy's quasi-experimental scope but is well suited to a structural-causal-model / DAG approach (e.g.
pathmc), the skill should say so and point the user there, rather than only returning "not implemented in CausalPy". A no-fit outcome is far more useful when it names the right neighbouring tool. Keep this at the level of intent: the skill should recognise when a sibling library is the better home and route to it. Exact wording and the boundary between the two are for the skill authors to define.Suggested acceptance criteria
choosing-causalpy-methodsbefore selecting a method.choosing-causalpy-methodsgives a correct route for a graded intervention at known times (Piecewise ITS), with the no-fit boundary re-scoped to genuine dose-response-curve cases.pathmc) when the problem is better served there, instead of ending at a bare "not implemented".