Skip to content

v1.9.0 Pass 4 — Stata expansion (stata-mcp + /stata-replication + audit-reproducibility) #121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pedrohcgs merged 1 commit into from
May 20, 2026
Merged

v1.9.0 Pass 4 — Stata expansion (stata-mcp + /stata-replication + audit-reproducibility) #121

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
179 changes: 179 additions & 0 deletions .claude/rules/stata-code-conventions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
---
paths:
- "**/*.do"
- "scripts/stata/**"
---

# Stata Code Conventions

**Reproducibility is the default, not a feature.** A `.do` file should be runnable from a clean shell with no manual intervention. Every script ends in the same state regardless of how many times it runs.

> This rule mirrors [`r-code-conventions.md`](r-code-conventions.md) for users whose pipelines are Stata-first. Forkers who use both R and Stata in the same project: both rules apply on their respective files.

## 1. Reproducibility scaffolding

Every `.do` file starts with the same header:

```stata
/*------------------------------------------------------------
File: NN_descriptive_name.do
Purpose: [one-sentence description]
Inputs: [path to inputs]
Outputs: [path to outputs]
Run order: Standalone | After NN_prior.do
------------------------------------------------------------*/

version 18 // pin Stata semantics
clear all
set more off
set seed 12345 // pin RNG seed for any random ops
set sortseed 12345 // pin sort stability across versions
cap log close
cap log close _all // belt-and-suspenders
log using "scripts/stata/_outputs/NN_log.smcl", replace
```

Why each line:

- **`version 18`** — explicit semantics. New Stata versions can silently change defaults (e.g., `reghdfe` clustering df-adjustment); pinning is the only defence.
- **`clear all`** — no leftover state from a prior session.
- **`set more off`** — scripts shouldn't pause for keystrokes.
- **`set seed` + `set sortseed`** — every random op + every `sort` is deterministic.
- **`cap log close _all`** — pre-emptively close any logs the previous session left open.
- **`log using ... , replace`** — capture stdout; `replace` ensures re-runs don't append.

End every `.do` file with:

```stata
log close
exit, clear STATA // explicit clean exit; useful in batch mode
```

## 2. Numbered pipeline

Scripts live in `scripts/stata/` and are numbered for run order:

```
scripts/stata/
├── 00_install.do # ssc install packages, set globals, paths
├── 01_clean.do # raw → cleaned panel
├── 02_descriptive.do # summary tables, balance, attrition
├── 03_analyze.do # main regression specs
├── 04_robustness.do # alt specs, sensitivity
├── 05_tables_figures.do # estout/esttab + graph export
└── 99_run_all.do # do "01_clean.do" / do "02_..." / ...
```

The 99-script is the **one-command reproduction**: `do scripts/stata/99_run_all.do` from the repo root produces every output the paper cites. AEA Data Editor checks this exact shape.

## 3. Outputs convention

All outputs land in `scripts/stata/_outputs/`:

```
scripts/stata/_outputs/
├── 01_log.smcl # captured stdout per script
├── clean_panel.dta # cleaned data
├── descriptives.csv # summary stats
├── main_results.tex # esttab → .tex for direct \input{} in paper
├── balance_table.tex
├── fig_eventstudy.pdf # graph export, vector
└── sessionInfo.txt # capture stata version + installed pkg versions
```

`sessionInfo.txt` is mandatory. Generate via:

```stata
* At end of 00_install.do (or via a dedicated sessioninfo subroutine):
log using "scripts/stata/_outputs/sessionInfo.txt", text replace
which estout
which reghdfe
which ivreg2
about
log close
```

This gives the AEA / referee / future-you the package versions actually used.

## 4. Tables (estout / esttab)

Use `esttab` for any table that appears in the paper. **Never hand-format a table in LaTeX** — the `\input{}` pattern means the table cell values come from the actual estimation:

```stata
quietly: reghdfe y x1 x2, absorb(unit time) cluster(unit)
eststo m1
quietly: reghdfe y x1 x2 controls, absorb(unit time) cluster(unit)
eststo m2

esttab m1 m2 using "scripts/stata/_outputs/tab_main.tex", replace ///
booktabs label /// use the variable labels you set
se(2) b(3) /// SE in parens, 3-decimal coeffs
star(* 0.10 ** 0.05 *** 0.01) /// significance convention
stats(N r2, fmt(%9.0fc %9.3f) labels("Observations" "R²")) ///
nonotes addnote("Robust SEs clustered at unit level.")
```

Then in the manuscript: `\input{scripts/stata/_outputs/tab_main.tex}` — table values update mechanically every time the .do file runs.

## 5. Significance-stars convention

The default is `* 0.10 ** 0.05 *** 0.01` (the econ convention; matches AER / QJE / JPE / ECMA defaults). Political science journals (APSR / AJPS / JOP) often use the same. Document the convention in the table note even though it's "obvious" — referees read the notes.

For one-tailed contexts (rare in published work), use `* 0.05 ** 0.025 *** 0.005` and explain why in the note. Default to two-tailed.

## 6. Clustering and SE conventions

- **Always cluster at the level of treatment assignment** (or the highest plausible level of dependence). Never use default `, robust` without justification.
- **`reghdfe` defaults to df-adjusted clustering** but check — Stata's `, cluster()` and `reghdfe ... , cluster()` use different df adjustments in some edge cases. The version pin at top of file is partial defence; explicit `, dofadj() ` is the rest.
- **Bootstrap clustering** for very small clusters (< 50 groups): use `cluster bootstrap` not `bootstrap, cluster()`.

## 7. Balance / attrition discipline

For every RCT or quasi-experimental design:

- **Balance table** via `iebaltab` (World Bank `ietoolkit` package). Don't reinvent.
- **Attrition table** — fraction missing per round, balanced by treatment status. Same `iebaltab` invocation with different outcome.
- **Manipulation checks** (survey experiments) — pass rate per arm. Document in the paper, not just the .do.

## 8. Figures (graph export)

```stata
graph export "scripts/stata/_outputs/fig_eventstudy.pdf", replace as(pdf)
graph export "scripts/stata/_outputs/fig_eventstudy.png", replace as(png) width(2000)
```

Both vector (PDF for the paper) and raster (PNG for slides). Don't rely on the auto-generated `.gph` — it's not portable across Stata versions.

## 9. Common Stata → R / Stata → AEA traps

| Trap | Fix |
|---|---|
| `reg y x, cluster(id)` without explicit df-adjustment | Use `reghdfe` for explicit df; document the adjustment in the table note |
| Bootstrap reps inconsistent across runs | `set seed` + `set sortseed` at top of file; use `bootstrap, reps(N) seed(X)` |
| `replace` modifying observed data | Always `gen new_var = ... ` and inspect before `drop` |
| `merge 1:1` without `assert` | `merge 1:1 id using foo, assert(3)` — fail loud on mismatched keys |
| `if` on missing values | Stata treats `.` as `+∞` in inequality comparisons. `if x > 5 & x != .` for non-missing-and-greater-than-5 |
| `egen sum(x)` deprecated | `egen total(x)` is the modern form; `egen sum()` still works but `bysort id: egen y = total(x)` is the safer pattern |

## 10. AEA Data Editor compliance

The [AEA Data Editor checklist](https://aeadataeditor.github.io/) requires:

- `README.md` at repo root describing data source, computational requirements, run instructions.
- A single command that reproduces all results (`do scripts/stata/99_run_all.do`).
- All scripts numbered and ordered.
- A separate `requirements.txt`-equivalent — for Stata, that's the `sessionInfo.txt` from §3 plus a list of `ssc install` commands in `00_install.do`.
- License (MIT / GPL / similar).
- No hard-coded paths — use globals or relative paths from repo root.

## Enforcement

- [`/stata-replication`](../skills/stata-replication/SKILL.md) is the analogue of `/data-analysis` for Stata. It emits .do files conforming to this convention.
- [`/audit-reproducibility`](../skills/audit-reproducibility/SKILL.md) handles Stata `.dta` outputs alongside R `.rds` (via `haven` or `pyreadstat`).
- [`/review-r`](../skills/review-r/SKILL.md) is R-specific; a Stata-equivalent is on the v1.9-backlog.

## Cross-references

- [`r-code-conventions.md`](r-code-conventions.md) — analogous discipline for R-first pipelines.
- [`replication-protocol.md`](replication-protocol.md) — tolerance contract that applies across R / Stata / Python.
- [stata-mcp on GitHub](https://github.com/SepineTam/stata-mcp) — the MCP server that lets Claude Code execute Stata `.do` files. Install via `claude mcp add stata-mcp --scope user -- uvx stata-mcp`.
18 changes: 17 additions & 1 deletion .claude/skills/audit-reproducibility/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Compare numeric claims in a manuscript (point estimates, standard errors, p-valu
## Inputs

- `$0` — path to the manuscript (`.tex`, `.qmd`, `.md`, `.pdf`). Required.
- `$1` — path to the outputs directory. Defaults to `scripts/R/_outputs/`. Can be `_targets/objects/`, a Stata `.do`-file log directory, etc.
- `$1` — path to the outputs directory. Defaults to `scripts/R/_outputs/`. Recognised alternatives: `scripts/stata/_outputs/` (Stata pipelines built by [`/stata-replication`](../stata-replication/SKILL.md)), `_targets/objects/` (R `targets` workflows), any directory the user-specified outputs live in.

## Workflow

Expand Down Expand Up @@ -153,6 +153,22 @@ Write `quality_reports/reproducibility_audit_[manuscript-name].md`:
- **Any FAIL:** exit 1, summary printed to stderr. This makes the skill usable as a `/commit` pre-commit gate — see `replication-protocol.md` for the enforcement pattern.
- **UNMATCHED > 0 (with 0 FAIL):** exit 0 with warning — user must manually review.

## Source-language coverage

The skill compares manuscript claims against outputs in three source-language ecosystems:

| Source | Default outputs dir | Read-output via | Common claim sources |
|---|---|---|---|
| **R** (default) | `scripts/R/_outputs/` | `readRDS()`, `arrow::read_parquet()`, `vroom::vroom()` | `.rds` / `.parquet` / `.csv` / `tinytable` `.tex` |
| **Stata** (v1.9.0) | `scripts/stata/_outputs/` | `haven::read_dta()` from R, or `pyreadstat.read_dta()` from Python | `.dta` / `esttab` `.tex` / `.smcl` log values |
| **Python** | `scripts/python/_outputs/` (or `_targets/`) | `pandas.read_parquet`, `pickle.load` | `.parquet` / `.pickle` / `.csv` |

**Stata-specific notes (v1.9.0):**

- `.dta` outputs are read via `haven::read_dta()` (R), `pyreadstat.read_dta()` (Python), or by parsing the corresponding `esttab` `.tex` if the table-cell value is what the manuscript cites.
- Manuscript cell `\input{scripts/stata/_outputs/tab_main.tex}` is the strongest provenance signal — the cell value comes mechanically from the .do file. Match the location in the `.tex` to the regression call in `03_analyze.do`.
- Clustering df adjustments can differ between `reghdfe` and base `reg, cluster()`. If a SE mismatches at the 2nd decimal, the tolerance in `replication-protocol.md` covers it; if it mismatches at the 1st decimal, investigate the df adjustment.

## Passport-mode (v1.9.0)

When `quality_reports/passports/<paper-slug>.yaml` exists, the skill operates in **passport mode**: instead of emitting a one-shot report, it **reads, updates, and rewrites** the passport file in place.
Expand Down
117 changes: 117 additions & 0 deletions .claude/skills/stata-replication/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
---
name: stata-replication
description: End-to-end Stata replication pipeline — scaffolds numbered `.do` files in `scripts/stata/`, executes them via the `stata-mcp` MCP server, captures logs and outputs to `scripts/stata/_outputs/`, and produces publication-ready tables (esttab) and figures (graph export). Mirrors `/data-analysis` for R-first projects. Use when user says "stata replication", "set up Stata pipeline", "scaffold the .do files", "run Stata analysis", "AEA replication package in Stata", or when a project's analysis language is Stata not R.
author: Claude Code Academic Workflow
version: 1.0.0
argument-hint: "[paper-or-data-pointer] [--from-r] [--no-execute]"
disable-model-invocation: true
allowed-tools: ["Read", "Write", "Edit", "Glob", "Grep", "Bash", "Task"]
---

# `/stata-replication` — Stata pipeline scaffold + execution

Build a complete Stata replication pipeline in `scripts/stata/`: numbered `.do` files following [`.claude/rules/stata-code-conventions.md`](../../rules/stata-code-conventions.md), executed via the [`stata-mcp`](https://github.com/SepineTam/stata-mcp) MCP server, with outputs landing in `scripts/stata/_outputs/`.

## When to use

- Your project's analysis language is **Stata** (not R). Common in econ field experiments, RCT studies, and any AEA submission where the original replication package is Stata.
- You're porting an R-first project to Stata for an AEA submission.
- You're adding a Stata robustness check to an R-first paper.
- You want a one-command reproduction: `do scripts/stata/99_run_all.do`.

## When NOT to use

- Your project is R-first. Use [`/data-analysis`](../data-analysis/SKILL.md).
- Your project is Python-first. Neither this skill nor `/data-analysis` is the right fit; consider extending the convention rule for Python or porting one of these skills.
- You're doing quick exploratory work. The numbered-pipeline scaffold is for replication packages, not scratch notebooks.

## Prerequisite: `stata-mcp` installed

This skill requires the `stata-mcp` MCP server. Install once per user:

```bash
claude mcp add stata-mcp --scope user -- uvx stata-mcp
```

The MCP server provides command-guarded Stata execution (refuses destructive operations like `!/shell/erase`), RAM monitoring, and Stata Language Server pairing. Maintained by SepineTam, 171 stars on GitHub as of 2026-05.

If `stata-mcp` is not installed, the skill halts at Phase 0 with installation instructions.

## Workflow

### Phase 0: Pre-flight

1. Verify `stata-mcp` is registered in the user's MCP configuration. If not → halt with install instructions.
2. Verify Stata is installed locally (the MCP server cannot run without it). Output stata version to confirm.
3. Confirm `scripts/stata/` directory exists or can be created.
4. Read [`.claude/rules/stata-code-conventions.md`](../../rules/stata-code-conventions.md) — every emitted `.do` file follows this convention.
5. If `--from-r` flag is set, locate the existing R pipeline at `scripts/R/` and use it as a translation source. Apply the Stata → R pitfalls table from `replication-protocol.md` in reverse.

### Phase 1: Scaffold the pipeline

Emit (or update) these files in `scripts/stata/`, each conforming to the header convention from `stata-code-conventions.md`:

```
scripts/stata/
├── 00_install.do # ssc install, set globals, paths, sessionInfo capture
├── 01_clean.do # raw → cleaned panel
├── 02_descriptive.do # summary tables, balance (iebaltab), attrition
├── 03_analyze.do # main regression specs (reghdfe / ivreg2 as needed)
├── 04_robustness.do # alt specs, sensitivity
├── 05_tables_figures.do # esttab .tex outputs + graph export PDFs
└── 99_run_all.do # do "01_clean.do" / do "02_..." / ...
```

If the paper or data source suggests specific specs (e.g., DiD with `reghdfe`, IV with `ivreg2`, RD with `rdrobust`), tailor `03_analyze.do` accordingly.

### Phase 2: Execute (unless `--no-execute`)

For each script in numbered order:

1. Dispatch to `stata-mcp` to execute the `.do` file.
2. Capture the log (Stata writes to `scripts/stata/_outputs/NN_log.smcl` per the header convention) and the resulting `.dta` / `.tex` / `.pdf` outputs.
3. If a script fails, halt — do NOT auto-fix unless the failure is trivial (typo flagged by Stata at parse time). For substantive failures (insufficient observations, singular matrices, missing covariates), surface to the user.

For long-running scripts (> 2 minutes), use the **Monitor tool** to stream stdout — same pattern documented in `/data-analysis` and `/audit-reproducibility`.

### Phase 3: Verify

1. Confirm every expected output exists in `scripts/stata/_outputs/`.
2. Check `sessionInfo.txt` was captured (package versions).
3. Run `/audit-reproducibility` if a manuscript exists — it now handles Stata `.dta` outputs via `haven`/`pyreadstat` (Pass 4.3).
4. Report scripts run, outputs produced, any warnings from Stata.

### Phase 4 (optional): R cross-check

If `--from-r` was set, run the R version of the same analysis (assumed to live at `scripts/R/`) and compare:

- Point estimates: should match to ~0.01 (per `replication-protocol.md` tolerance).
- Standard errors: should match to ~0.05 (clustering df adjustments can differ slightly between Stata and R).
- Sample sizes: must match exactly.

Discrepancies are surfaced for the user to investigate — typical culprits: clustering df, default options (logit vs probit for PS), bootstrap seed handling.

## Companion skills

- [`/data-analysis`](../data-analysis/SKILL.md) — R analogue. Same pipeline shape, different language.
- [`/audit-reproducibility`](../audit-reproducibility/SKILL.md) — reads both `.rds` and `.dta` outputs. Cross-checks manuscript claims against the produced values. Updated in v1.9.0 to handle Stata outputs.
- [`/review-paper`](../review-paper/SKILL.md) — if the paper exists and cites tables/figures produced by this pipeline, `/review-paper` auto-invokes `/audit-reproducibility` (per `cross-artifact-review.md`).

## Anti-patterns

- **Hand-editing `.dta` files.** Never. All transformations happen via the `.do` files; `.dta` outputs are derived and reproducible.
- **Skipping the `99_run_all.do`.** This is the AEA-mandated one-command entry point. Build it even for small projects.
- **Using `, robust` by default.** Use `, cluster(id)` at the appropriate level — see `stata-code-conventions.md` §6.
- **Hand-formatting tables in LaTeX.** Use `esttab` and `\input{}` — see `stata-code-conventions.md` §4.
- **Pinning Stata version in only one .do file.** Every `.do` file starts with `version 18` per the convention.

## Cross-references

- [`.claude/rules/stata-code-conventions.md`](../../rules/stata-code-conventions.md) — the discipline contract.
- [`.claude/rules/replication-protocol.md`](../../rules/replication-protocol.md) — tolerance thresholds (applies across R / Stata / Python).
- [stata-mcp on GitHub](https://github.com/SepineTam/stata-mcp) — the MCP server this skill depends on.
- [AEA Data Editor checklist](https://aeadataeditor.github.io/) — replication-package standards.

## Long-running fits / batch reruns: use the Monitor tool (Apr 2026)

Long Stata fits (multi-hour bootstrap with `cluster bootstrap`, large `reghdfe` with millions of observations, simulation studies) should be background-launched and tailed with the Monitor tool — same pattern as `/data-analysis` and `/audit-reproducibility` for R / Python. The .do file logs to SMCL; the Monitor tool follows stderr so Claude can react to errors mid-stream.
Loading