Add analytic SE and CI for IPW estimator via influence function#1641
Open
genrichez wants to merge 3 commits into
Open
Add analytic SE and CI for IPW estimator via influence function#1641genrichez wants to merge 3 commits into
genrichez wants to merge 3 commits into
Conversation
genrichez
force-pushed
the
feature/ipw-analytic-se
branch
from
July 4, 2026 08:28
bcc6bef to
2728277
Compare
This was referenced Jul 4, 2026
Open
This comment has been minimized.
This comment has been minimized.
Author
|
Thanks for checking it out, fixed in the latest commit along with the other suggestions |
The PropensityScoreWeightingEstimator had no _estimate_std_error or _estimate_confidence_intervals, so CI requests silently fell back to bootstrap. This adds the closed-form influence-function-based variance from Lunceford & Davidian (2004). All weighting schemes in this estimator are ratio (Hajek) estimators because estimate_effect divides by sum of weights. The IF accounts for this ratio structure. Covers ATE, ATT, and ATC targets. Includes 13 new tests validating against bootstrap, checking sqrt(n) scaling, coverage, and all scheme/target combinations. Signed-off-by: genrichez <22434764+genrichez@users.noreply.github.com>
- Fix ATT control-arm IF normalizer: use mean((1-T)*e/(1-e)) instead of mean(T). The control counterfactual is a ratio with its own denominator, not the treatment proportion. - Apply the same fix to ATC treated-arm IF for consistency. - Add hasattr guard in _estimate_std_error so calling it before estimate_effect gives a clear RuntimeError instead of AttributeError. - Add comment noting that DataFrame target_units uses the population ATE influence function (per-unit custom IF not yet supported). - Add comment explaining why we use sum(IF^2)/n^2 (asymptotically equivalent to var(IF)/n since IF has mean zero by construction). Signed-off-by: genrichez <meltsefon1@gmail.com> Signed-off-by: genrichez <22434764+genrichez@users.noreply.github.com>
genrichez
force-pushed
the
feature/ipw-analytic-se
branch
from
July 5, 2026 11:19
fd37f48 to
17bf69e
Compare
Contributor
|
Note The pull request was not created — a fallback review issue was created instead due to protected file changes: #1646 🤖 This is an automated response from Repo Assist. Thanks for the swift fixes,
The math is solid across all three target units and the tests are comprehensive. This PR is in great shape! 🎉
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds analytic standard errors and confidence intervals for PropensityScoreWeightingEstimator using the influence function approach from Lunceford & Davidian (2004).
Problem
The IPW estimator has no _estimate_std_error or _estimate_confidence_intervals. When a user asks for CIs, the base class silently falls back to a bootstrap. This is slow and the user never asked for it.
Solution
The estimator computes a Hajek (ratio) weighted mean in each treatment arm. The influence function for this is:
IF_i = (T_i/e_i) * (Y_i - mu1) / (sum(T/e)/n) - ((1-T_i)/(1-e_i)) * (Y_i - mu0) / (sum((1-T)/(1-e))/n)
Then SE = sqrt(Var(IF) / n) and CI = estimate +/- z * SE.
I added:
_compute_influence_function() - computes the per-unit IF for ATE, ATT, and ATC
_estimate_std_error() - returns sqrt(Var(IF)/n)
_estimate_confidence_intervals() - returns estimate +/- z * SE
The stored arrays (_ipw_T, _ipw_Y, _ipw_ps) are saved during estimate_effect so the IF can be computed later without re-fitting.
Important detail: Hajek vs Horvitz-Thompson
All weighting schemes here are actually ratio estimators because the code divides by sum_dy_weights:
est = data["d_y"].sum() / sum_dy_weights - data["dbar_y"].sum() / sum_dbary_weights
This is sum(T*Y/e)/sum(T/e) - ..., which is the Hajek form. Using the simpler Horvitz-Thompson IF (which assumes (1/n)*sum(...) without normalization) gives SE about 3x too large. The implementation uses the correct ratio-linearized IF for all schemes.
Tests
Added 13 new tests covering:
Known behavior
The IF-based SE is slightly conservative (about 30% wider than the true sampling variability) because it treats propensity scores as known. This is standard - same behavior as R's WeightIt and Stata's teffects ipw. It means the CI has over-coverage (safe) rather than under-coverage.
Reference
Lunceford, J.K. & Davidian, M. (2004). Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Statistics in Medicine, 23(19), 2937-2960.
Closes #1640