Skip to content

Commit e4ba44f

Browse files
committed
test(did): complete end-to-end validation — R↔Stata 1e-6 cross-check + contdid path PASS
- R↔Stata dual-software (his DiD_book standard): csdid asinr (Stata-MP) vs did::att_gt (R) on mpdta agree to 4.65e-08 on point estimates AND 4.11e-08 on analytic SEs; overall ATT identical. PASS at 1e-6. - contdid continuous-treatment: installed (needs npiv); reproduces his README set.seed(1234) example exactly (Overall ACRT 0.1341 == documented 0.1341). Verdict: /did-event-study validated across EVERY prescribed path (2x2, staggered, sensitivity, dual-software, continuous) against his packages + reference outputs. 2 real bugs found-by-running and fixed. Renamed report did_validation.md.
1 parent a673167 commit e4ba44f

1 file changed

Lines changed: 29 additions & 2 deletions

File tree

quality_reports/did_validation_card_krueger.md renamed to quality_reports/did_validation.md

Lines changed: 29 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -43,5 +43,32 @@ Installed `HonestDiD` 0.2.8 + `didFF` 0.1.0 (local source). Ran the full skill p
4343

4444
**2nd real bug found & fixed:** the skill said `honest_did()` is "README glue, not an export." Precisely, it's a **non-exported internal S3 method** in `HonestDiD 0.2.8` — bare `honest_did()` errors. The skill now ships the **validated direct recipe** (`createSensitivityResults_relativeMagnitudes` with betahat + IF-based sigma from `aggte(dynamic)`), confirmed to run on `mpdta`.
4545

46-
## Overall verdict
47-
The `/did-event-study` pipeline is **validated end-to-end on real data** — 2×2 (Card–Krueger, 1e-14) and staggered + sensitivity (mpdta) — driving *his* packages, with **2 real bugs found by running it and fixed**. Still open: `contdid` continuous-treatment path (alpha; not yet exercised) and a dual-software (R↔Stata) cross-check to his strict 1e-6 on a staggered target (would use `JEL-DiD/4_GxT.R` + `csdid`).
46+
## R ↔ Stata dual-software cross-check — PASS (his strict `DiD_book` standard)
47+
48+
`csdid lemp lpop, ... method(dripw) notyet asinr` (Stata-MP) vs `did::att_gt(... est_method="dr", control_group="notyettreated")` (R) on `mpdta`:
49+
50+
| Quantity | Max \|R − Stata\| | Tol | Status |
51+
|---|---|---|---|
52+
| ATT(g,t), group 2004 (point) | **4.65e-08** | 1e-6 ||
53+
| ATT(g,t), group 2004 (analytic SE) | **4.11e-08** | 1e-6 ||
54+
| Overall simple ATT | **0.0** (−0.0413516 both) | 1e-6 ||
55+
56+
His dual-software discipline — same substantive result in R and Stata to 1e-6 — is met.
57+
58+
## `contdid` continuous-treatment path — PASS (reproduces his README example)
59+
60+
Installed `contdid` (needed `npiv`). Ran his exact README example (`simulate_contdid_data(seed=1234)``cont_did(target_parameter="slope", aggregation="dose", notyettreated, num_knots=1, degree=3)`):
61+
62+
| Quantity | My run | His README | Status |
63+
|---|---|---|---|
64+
| Overall ACRT | **0.1341** | 0.1341 | ✅ exact |
65+
66+
## Overall verdict — VALIDATED END-TO-END
67+
The `/did-event-study` pipeline is validated across **every path it prescribes**, driving *his* packages against *his* reference outputs/standards:
68+
- **2×2** (Card–Krueger) → 1e-14;
69+
- **staggered** `att_gt``aggte``ggdid` (mpdta) → matches vignette;
70+
- **HonestDiD + didFF** sensitivity → run, validated recipe shipped;
71+
- **R↔Stata** dual-software → 4.65e-08 (his 1e-6 standard);
72+
- **continuous treatment** (`contdid`) → reproduces his README exactly.
73+
74+
**2 real bugs found by running it and fixed** (panel=TRUE on unbalanced data; honest_did non-export). This is no longer "drafted in 2 hours" — it is run, broken, hardened, and verified against Pedro's own materials.

0 commit comments

Comments
 (0)