Skip to content

Commit 29eb364

Browse files
TomeHirataclaude
andcommitted
Restructure paper to comply with JOSS submission guidelines
- Add required "State of the Field" section comparing with SciPy, DoWhy, EconML, and causal-curve packages - Add required "Software Design" section explaining class architecture - Add required "Research Impact Statement" with ICML publications and industry usage - Add required "AI Usage Disclosure" section - Remove detailed "Features" section (API docs belong in documentation) - Add SciPy and causal-curve references to bibliography - Ensure word count is within 750-1750 range (776 words) - Add VanderPlas to typos allowlist (author name in SciPy citation) References: - JOSS submission guidelines: https://joss.readthedocs.io/en/latest/paper.html - JOSS review checklist: https://github.com/openjournals/joss/blob/main/docs/review_checklist.md 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent b312938 commit 29eb364

3 files changed

Lines changed: 57 additions & 37 deletions

File tree

_typos.toml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,7 @@
22
# Allow dataset-specific column names from Hillstrom dataset
33
womens = "womens"
44
mens = "mens"
5+
6+
[default.extend-identifiers]
7+
# Author name in SciPy citation
8+
VanderPlas = "VanderPlas"

paper.bib

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,3 +93,36 @@ @inproceedings{econml
9393
year={2021},
9494
organization={ACM}
9595
}
96+
97+
@ARTICLE{2020SciPy-NMeth,
98+
author = {Virtanen, Pauli and Gommers, Ralf and Oliphant, Travis E. and
99+
Haberland, Matt and Reddy, Tyler and Cournapeau, David and
100+
Burovski, Evgeni and Peterson, Pearu and Weckesser, Warren and
101+
Bright, Jonathan and van der Walt, St{\'e}fan J. and
102+
Brett, Matthew and Wilson, Joshua and Millman, K. Jarrod and
103+
Mayorov, Nikolay and Nelson, Andrew R. J. and Jones, Eric and
104+
Kern, Robert and Larson, Eric and Carey, C J and
105+
Polat, {\.I}lhan and Feng, Yu and Moore, Eric W. and
106+
VanderPlas, Jake and Laxalde, Denis and Perktold, Josef and
107+
Cimrman, Robert and Henriksen, Ian and Quintero, E. A. and
108+
Harris, Charles R. and Archibald, Anne M. and
109+
Ribeiro, Ant{\^o}nio H. and Pedregosa, Fabian and
110+
van Mulbregt, Paul and {SciPy 1.0 Contributors}},
111+
title = {{SciPy} 1.0: Fundamental Algorithms for Scientific Computing in Python},
112+
journal = {Nature Methods},
113+
year = {2020},
114+
volume = {17},
115+
pages = {261--272},
116+
doi = {10.1038/s41592-019-0686-2}
117+
}
118+
119+
@article{kobrosly2020causalcurve,
120+
title={causal-curve: A Python Causal Inference Package to Estimate Causal Dose-Response Curves},
121+
author={Kobrosly, Roni W.},
122+
journal={Journal of Open Source Software},
123+
volume={5},
124+
number={52},
125+
pages={2523},
126+
year={2020},
127+
doi={10.21105/joss.02523}
128+
}

paper.md

Lines changed: 20 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -34,59 +34,42 @@ bibliography: paper.bib
3434

3535
# Summary
3636

37-
`dte_adj` is a Python package designed for estimating distributional treatment effects (DTEs) in randomized experiments. Unlike traditional approaches that focus on average treatment effects, `dte_adj` enables researchers to analyze the full distributional impact of interventions across different outcome levels. The package implements machine learning-enhanced regression adjustment methods to achieve variance reduction, making distributional effect estimation more precise and computationally efficient. It supports multiple experimental designs including simple randomization, covariate-adaptive randomization (CAR), and local distributional treatment effect (LDTE) estimation. The package provides a scikit-learn compatible API and comprehensive functionality for computing distribution functions, probability treatment effects, and quantile treatment effects with confidence intervals.
37+
`dte_adj` is a Python package for estimating distributional treatment effects (DTEs) in randomized experiments (RCTs, also known as A/B tests). Unlike traditional approaches that focus on average treatment effects, `dte_adj` enables researchers to analyze the full distributional impact of interventions across different outcome levels. The package implements machine learning-enhanced regression adjustment methods for variance reduction, supports multiple experimental designs including simple randomization, covariate-adaptive randomization, and settings with imperfect compliance, and provides a scikit-learn compatible API with comprehensive functionality for computing distribution functions, probability treatment effects, and quantile treatment effects with confidence intervals.
3838

3939
# Statement of Need
4040

41-
Randomized experiments (RCTs, also known as A/B tests) have been fundamental to scientific inquiry since the pioneering work of @fisher1935design, providing the gold standard for causal inference. While most experimental analyses focus on average treatment effects (ATEs), many research questions require understanding how treatments affect the entire distribution of outcomes, not just the mean. Distributional treatment effects (DTEs) capture these richer patterns, revealing heterogeneous impacts across different outcome levels that averages can mask.
41+
Randomized experiments have been fundamental to scientific inquiry since @fisher1935design, providing the gold standard for causal inference. While most experimental analyses focus on average treatment effects (ATEs), many research questions require understanding how treatments affect the entire distribution of outcomes. Distributional treatment effects (DTEs) capture these richer patterns, revealing heterogeneous impacts across different outcome levels that averages can mask. For example, a policy intervention might have no effect on average income while substantially reducing poverty rates at lower quantiles, or a medical treatment might benefit patients at the tails of the distribution differently than those near the median.
4242

43-
Despite the growing importance of distributional analysis in fields ranging from economics to medicine, the Python ecosystem lacks comprehensive tools for DTE estimation. While SciPy provides basic empirical cumulative distribution functions, it offers no specialized functionality for treatment effect estimation, variance reduction, or confidence interval construction in experimental settings. Existing Python packages for causal inference, such as DoWhy [@dowhy] and EconML [@econml], focus primarily on average treatment effects and conditional average treatment effects (CATE), with EconML incorporating machine learning for heterogeneous treatment effect estimation. However, these packages do not address distributional treatment effects, which capture how treatments affect the entire outcome distribution rather than just the mean. Existing R packages like `RDDtools` focus on regression discontinuity rather than randomized experiments, and lack modern machine learning integration.
43+
Despite the growing importance of distributional analysis in economics, medicine, and technology, the Python ecosystem lacks comprehensive tools for DTE estimation with modern variance reduction techniques. Researchers often resort to basic empirical CDFs or manual implementations that lack statistical rigor. `dte_adj` fills this gap by providing a unified framework for distributional treatment effect analysis that integrates state-of-the-art machine learning methods for improved precision, rigorous confidence interval construction, and support for complex experimental designs.
4444

45-
`dte_adj` addresses this gap by providing a comprehensive Python framework for distributional treatment effect analysis. The package implements state-of-the-art variance reduction techniques using machine learning models for regression adjustment [@byambadalai2024estimatingdistributionaltreatmenteffects], enabling more precise DTE estimates with smaller sample sizes. It supports multiple experimental designs including covariate-adaptive randomization [@byambadalai2025efficientestimationdistributionaltreatment] and local treatment effects, with a scikit-learn [@scikit-learn] compatible API that integrates seamlessly into existing machine learning workflows. This makes advanced distributional analysis accessible to the broader Python research community, supporting more nuanced causal inference in experimental studies.
45+
# State of the Field
4646

47-
# Features
47+
Several Python packages address causal inference, but none focus on distributional treatment effects with machine learning-based variance reduction:
4848

49-
`dte_adj` provides a comprehensive suite of tools for distributional treatment effect analysis:
49+
- **SciPy** [@2020SciPy-NMeth]: Provides basic empirical cumulative distribution functions but offers no functionality for treatment effect estimation or confidence interval construction in experimental settings.
50+
- **DoWhy** [@dowhy]: Focuses on causal graph-based inference and average treatment effects, without distributional analysis capabilities.
51+
- **EconML** [@econml]: Incorporates machine learning for heterogeneous treatment effect estimation (CATE) but does not address distributional effects.
52+
- **causal-curve** [@kobrosly2020causalcurve]: Estimates dose-response curves but targets continuous treatments rather than distributional outcomes.
5053

51-
## Estimator Classes
54+
In the R ecosystem, packages like `qte` provide quantile treatment effect estimation but lack machine learning integration for variance reduction. `dte_adj` uniquely combines: (1) distributional treatment effect estimation across the full outcome distribution, (2) machine learning-enhanced regression adjustment for precision gains, and (3) support for multiple experimental designs including covariate-adaptive randomization and imperfect compliance settings.
5255

53-
The package implements multiple estimator classes following a hierarchical design pattern:
56+
# Software Design
5457

55-
**Simple Randomization Estimators:**
56-
- `SimpleDistributionEstimator`: Basic empirical distribution function estimator for simple randomized experiments
57-
- `AdjustedDistributionEstimator`: Machine learning-enhanced estimator with regression adjustment for variance reduction
58+
`dte_adj` follows a class-based architecture with a template method pattern, where a base class defines the algorithm structure and subclasses implement design-specific computations:
5859

59-
**Stratified Estimators (for Covariate-Adaptive Randomization):**
60-
- `SimpleStratifiedDistributionEstimator`: Handles stratified block randomization designs
61-
- `AdjustedStratifiedDistributionEstimator`: Combines stratification with ML-based variance reduction
60+
- **`SimpleDistributionEstimator`** and **`AdjustedDistributionEstimator`**: For simple randomized experiments, implementing methods from @byambadalai2024estimatingdistributionaltreatmenteffects.
61+
- **`SimpleStratifiedDistributionEstimator`** and **`AdjustedStratifiedDistributionEstimator`**: For covariate-adaptive randomization designs, implementing methods from @byambadalai2025efficientestimationdistributionaltreatment.
62+
- **`SimpleLocalDistributionEstimator`** and **`AdjustedLocalDistributionEstimator`**: For settings with imperfect compliance, implementing methods from @byambadalai2025imperfectcompliance.
6263

63-
**Local Distribution Estimators (for Imperfect Compliance):**
64-
- `SimpleLocalDistributionEstimator`: Estimates local distributional treatment effects (LDTE) for settings with imperfect compliance
65-
- `AdjustedLocalDistributionEstimator`: LDTE estimation with ML adjustment for improved precision [@byambadalai2025imperfectcompliance]
64+
All estimators implement a consistent API with three primary methods: `predict_dte()` for distributional treatment effects, `predict_pte()` for probability treatment effects over intervals, and `predict_qte()` for quantile treatment effects. The adjusted estimators use K-fold cross-fitting to prevent overfitting and support both single-task and multi-task learning modes [@hirata2025efficientscalableestimationdistributional] for computational efficiency. Bootstrap methods provide confidence intervals with multiple variance estimation approaches.
6665

67-
## Core Methods
66+
# Research Impact Statement
6867

69-
All estimators implement a consistent API with three primary methods:
68+
The methods implemented in `dte_adj` have been published at top machine learning venues: ICML 2024 [@byambadalai2024estimatingdistributionaltreatmenteffects] and ICML 2025 [@byambadalai2025efficientestimationdistributionaltreatment]. The package has been used internally at CyberAgent, Inc. for analyzing A/B tests where distributional impacts are critical, such as evaluating interventions on user engagement metrics where tail behavior matters more than averages. The documentation includes tutorials demonstrating applications to the Hillstrom email marketing dataset and the Oregon Health Insurance Experiment, facilitating adoption by researchers in economics, marketing, and healthcare.
7069

71-
- `predict_dte()`: Computes Distributional Treatment Effects $DTE_{w, w'}(y) := F_{Y(w)}(y) - F_{Y(w')}(y)$, where $F_{Y(w)}(y)$ represents the cumulative distribution function for treatment $w$ at outcome level $y$.
70+
# AI Usage Disclosure
7271

73-
- `predict_pte()`: Computes Probability Treatment Effects over specified intervals, measuring differences in probability mass between treatment groups.
74-
75-
- `predict_qte()`: Computes Quantile Treatment Effects $QTE_{w, w'}(\tau) := F_{Y(w)}^{-1}(\tau) - F_{Y(w')}^{-1}(\tau)$, comparing quantiles across treatments.
76-
77-
## Advanced Features
78-
79-
**Multi-task Learning:** The package supports multi-task neural networks (`is_multi_task=True`) for computational efficiency when analyzing many outcome locations simultaneously [@hirata2025efficientscalableestimationdistributional].
80-
81-
**Cross-fitting:** Adjusted estimators use K-fold cross-fitting to prevent overfitting in machine learning models, ensuring robust treatment effect estimates.
82-
83-
**Confidence Intervals:** Built-in bootstrap methods provide confidence intervals with multiple variance estimation approaches (`moment`, `simple`, `uniform`).
84-
85-
**Visualization:** The `dte_adj.plot` module enables easy plotting of treatment effects and confidence bands.
86-
87-
![DTE](docs/source/_static/dte_moment.png)
88-
![PTE](docs/source/_static/pte_empirical.png)
89-
![QTE](docs/source/_static/qte.png)
72+
Generative AI tools (Claude) were used to assist with documentation writing and code review during development. All AI-generated content was reviewed and validated by the human authors.
9073

9174
# Acknowledgements
9275

0 commit comments

Comments
 (0)