JOSS submission (#17)

TomeHirata · claude · web-flow · commit 754bade163ba · 2026-04-20T14:47:33.000+09:00
* docs: draft for JOSS * yodate paper.md * Address PR review comments and update for latest codebase - Add "(RCTs, also known as A/B tests)" clarification per reviewer feedback - Add DoWhy and EconML comparison in Statement of Need section - Add imperfect compliance paper reference for LDTE estimators - Update CAR paper to ICML'25 proceedings format with updated author list - Update multi-task learning paper author list (add Shunsuke Uto) - Fix Fisher citation key to match bibliography 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Restructure paper to comply with JOSS submission guidelines - Add required "State of the Field" section comparing with SciPy, DoWhy, EconML, and causal-curve packages - Add required "Software Design" section explaining class architecture - Add required "Research Impact Statement" with ICML publications and industry usage - Add required "AI Usage Disclosure" section - Remove detailed "Features" section (API docs belong in documentation) - Add SciPy and causal-curve references to bibliography - Ensure word count is within 750-1750 range (776 words) - Add VanderPlas to typos allowlist (author name in SciPy citation) References: - JOSS submission guidelines: https://joss.readthedocs.io/en/latest/paper.html - JOSS review checklist: https://github.com/openjournals/joss/blob/main/docs/review_checklist.md 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Address Copilot review comments - Fix author metadata: make Tomu Hirata the single corresponding author and remove conflicting equal-contrib flag - Fix NumPy citation: replace Unicode en-dash (–) with BibTeX format (--) for compatibility across citation pipelines 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Move paper to paper/ subdirectory and add tutorial figures - Move paper.md and paper.bib into paper/ to avoid top-level clutter - Add hillstorm_dte.png and oregon_ldte_costs_comparison.png from tutorials - Add figure captions showing simple vs ML-adjusted comparison for both the Hillstrom email campaign and Oregon Health Insurance Experiment - Cross-reference figures in Research Impact Statement - Add dataset citations (Hillstrom 2008, Finkelstein et al. 2012) - Add GitHub Action (draft-pdf.yml) to auto-build JOSS PDF on push using paper-path: paper/paper.md Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Pin GitHub Actions to commit SHAs for supply chain security Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Update Databricks affiliation to Databricks, Inc., United States Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add explicit permissions to draft-pdf workflow Addresses CodeQL security alert: restrict GITHUB_TOKEN to read-only contents access following least-privilege principle. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix figure paths for inara PDF build inara/pandoc runs from the repo root, so relative image paths in paper/paper.md resolve against root, not paper/. Copy PNGs to root before the build step so pandoc finds them. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix multiply-defined LaTeX labels for figures Pandoc extracts \label{} from caption text and also auto-generates a figure label, causing the label to be defined twice. Remove explicit \label{} from captions and replace \autoref{} with plain text references. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix figure 1 filename: hillstorm_dte.png (not hillstrom) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Remove unnecessary cp step in draft-pdf workflow inara CDs into the paper/ directory before running pandoc, so images in paper/ are already on the resource path. No copying needed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add citations per reviewer feedback - Add Econometric Reviews publication (Oka et al. 2025) - Add NeurIPS 2025 reference for imperfect compliance paper - Add ABEMA field experiment paper (Yasui et al. 2026, Japanese Economic Review) as industry application evidence Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
diff --git a/.github/workflows/draft-pdf.yml b/.github/workflows/draft-pdf.yml
@@ -0,0 +1,22 @@
+on: [push]
+
+permissions:
+  contents: read
+
+jobs:
+  paper:
+    runs-on: ubuntu-latest
+    name: Paper Draft
+    steps:
+      - name: Checkout
+        uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5  # v4
+      - name: Build draft PDF
+        uses: openjournals/openjournals-draft-action@85a18372e48f551d8af9ddb7a747de685fbbb01c  # master
+        with:
+          journal: joss
+          paper-path: paper/paper.md
+      - name: Upload
+        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02  # v4
+        with:
+          name: paper
+          path: paper/paper.pdf
diff --git a/_typos.toml b/_typos.toml
@@ -2,3 +2,7 @@
 # Allow dataset-specific column names from Hillstrom dataset
 womens = "womens"
 mens = "mens"
+
+[default.extend-identifiers]
+# Author name in SciPy citation
+VanderPlas = "VanderPlas"
diff --git a/paper/hillstorm_dte.png b/paper/hillstorm_dte.png
diff --git a/paper/oregon_ldte_costs_comparison.png b/paper/oregon_ldte_costs_comparison.png
diff --git a/paper/paper.bib b/paper/paper.bib
@@ -0,0 +1,162 @@
+@misc{byambadalai2024estimatingdistributionaltreatmenteffects,
+      title={Estimating Distributional Treatment Effects in Randomized Experiments: Machine Learning for Variance Reduction},
+      author={Undral Byambadalai and Tatsushi Oka and Shota Yasui},
+      year={2024},
+      eprint={2407.16037},
+      archivePrefix={arXiv},
+      primaryClass={econ.EM},
+      url={https://arxiv.org/abs/2407.16037},
+}
+
+@book{fisher1935design,
+  title={The Design of Experiments},
+  author={Fisher, Ronald A.},
+  year={1935},
+  publisher={Oliver and Boyd}
+}
+
+@ARTICLE{2020NumPy-Array,
+  author  = {Harris, Charles R. and Millman, K. Jarrod and
+            van der Walt, Stéfan J and Gommers, Ralf and
+            Virtanen, Pauli and Cournapeau, David and
+            Wieser, Eric and Taylor, Julian and Berg, Sebastian and
+            Smith, Nathaniel J. and Kern, Robert and Picus, Matti and
+            Hoyer, Stephan and van Kerkwijk, Marten H. and
+            Brett, Matthew and Haldane, Allan and
+            Fernández del Río, Jaime and Wiebe, Mark and
+            Peterson, Pearu and Gérard-Marchant, Pierre and
+            Sheppard, Kevin and Reddy, Tyler and Weckesser, Warren and
+            Abbasi, Hameer and Gohlke, Christoph and
+            Oliphant, Travis E.},
+  title   = {Array programming with {NumPy}},
+  journal = {Nature},
+  year    = {2020},
+  volume  = {585},
+  pages   = {357--362},
+  doi     = {10.1038/s41586-020-2649-2}
+}
+
+@article{scikit-learn,
+  title={Scikit-learn: Machine Learning in {P}ython},
+  author={Pedregosa, F. and Varoquaux, G. and Gramfort, A. and Michel, V.
+          and Thirion, B. and Grisel, O. and Blondel, M. and Prettenhofer, P.
+          and Weiss, R. and Dubourg, V. and Vanderplas, J. and Passos, A. and
+          Cournapeau, D. and Brucher, M. and Perrot, M. and Duchesnay, E.},
+  journal={Journal of Machine Learning Research},
+  volume={12},
+  pages={2825--2830},
+  year={2011}
+}
+
+@inproceedings{byambadalai2025efficientestimationdistributionaltreatment,
+      title={On Efficient Estimation of Distributional Treatment Effects under Covariate-Adaptive Randomization},
+      author={Undral Byambadalai and Tomu Hirata and Tatsushi Oka and Shota Yasui},
+      booktitle={Proceedings of the 42nd International Conference on Machine Learning},
+      year={2025},
+      series={ICML'25},
+      url={https://arxiv.org/abs/2506.05945}
+}
+
+@misc{hirata2025efficientscalableestimationdistributional,
+      title={Efficient and Scalable Estimation of Distributional Treatment Effects with Multi-Task Neural Networks},
+      author={Tomu Hirata and Undral Byambadalai and Tatsushi Oka and Shota Yasui and Shunsuke Uto},
+      year={2025},
+      eprint={2507.07738},
+      archivePrefix={arXiv},
+      primaryClass={econ.EM},
+      url={https://arxiv.org/abs/2507.07738}
+}
+
+@misc{byambadalai2025imperfectcompliance,
+      title={Beyond the Average: Distributional Causal Inference under Imperfect Compliance},
+      author={Undral Byambadalai and Tomu Hirata and Tatsushi Oka and Shota Yasui},
+      year={2025},
+      eprint={2509.15594},
+      archivePrefix={arXiv},
+      primaryClass={econ.EM},
+      url={https://arxiv.org/abs/2509.15594}
+}
+
+@article{oka2025regression,
+  title={Regression Adjustment for Estimating Distributional Treatment Effects in Randomized Controlled Trials},
+  author={Oka, Tatsushi and Yasui, Shota and Hayakawa, Yuta and Byambadalai, Undral},
+  journal={Econometric Reviews},
+  year={2025},
+  doi={10.1080/07474938.2025.2538843}
+}
+
+@article{yasui2026abema,
+  title={Distributional Treatment Effects of Content Promotion: Evidence from an {ABEMA} Field Experiment},
+  author={Yasui, Shota and Oka, Tatsushi and Byambadalai, Undral and Oishi, Yuki},
+  journal={The Japanese Economic Review},
+  year={2026},
+  doi={10.1007/s42973-026-00234-y}
+}
+
+@article{dowhy,
+  title={DoWhy: An End-to-End Library for Causal Inference},
+  author={Sharma, Amit and Kiciman, Emre},
+  journal={arXiv preprint arXiv:2011.04216},
+  year={2020},
+  url={https://arxiv.org/abs/2011.04216}
+}
+
+@inproceedings{econml,
+  title={Causal Inference and Machine Learning in Practice with {EconML} and {CausalML}: Industrial Use Cases at {Microsoft}, {TripAdvisor}, {Uber}},
+  author={Battocchi, Keith and Dillon, Eleanor and Hei, Maggie and Lewis, Greg and Ling, Miruna and Rao, Jing and Shyr, Dennis and Syrgkanis, Vasilis},
+  booktitle={Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery \& Data Mining},
+  pages={4072--4073},
+  year={2021},
+  organization={ACM}
+}
+
+@ARTICLE{2020SciPy-NMeth,
+  author  = {Virtanen, Pauli and Gommers, Ralf and Oliphant, Travis E. and
+            Haberland, Matt and Reddy, Tyler and Cournapeau, David and
+            Burovski, Evgeni and Peterson, Pearu and Weckesser, Warren and
+            Bright, Jonathan and van der Walt, St{\'e}fan J. and
+            Brett, Matthew and Wilson, Joshua and Millman, K. Jarrod and
+            Mayorov, Nikolay and Nelson, Andrew R. J. and Jones, Eric and
+            Kern, Robert and Larson, Eric and Carey, C J and
+            Polat, {\.I}lhan and Feng, Yu and Moore, Eric W. and
+            VanderPlas, Jake and Laxalde, Denis and Perktold, Josef and
+            Cimrman, Robert and Henriksen, Ian and Quintero, E. A. and
+            Harris, Charles R. and Archibald, Anne M. and
+            Ribeiro, Ant{\^o}nio H. and Pedregosa, Fabian and
+            van Mulbregt, Paul and {SciPy 1.0 Contributors}},
+  title   = {{SciPy} 1.0: Fundamental Algorithms for Scientific Computing in Python},
+  journal = {Nature Methods},
+  year    = {2020},
+  volume  = {17},
+  pages   = {261--272},
+  doi     = {10.1038/s41592-019-0686-2}
+}
+
+@misc{hillstrom2008,
+  title={The MineThatData E-Mail Analytics And Data Mining Challenge},
+  author={Hillstrom, Kevin},
+  year={2008},
+  url={https://blog.minethatdata.com/2008/03/minethatdata-e-mail-analytics-and-data.html}
+}
+
+@article{finkelstein2012,
+  title={The Oregon Health Insurance Experiment: Evidence from the First Year},
+  author={Finkelstein, Amy and Taubman, Sarah and Wright, Bill and Bernstein, Mira and Gruber, Jonathan and Newhouse, Joseph P. and Allen, Heidi and Baicker, Katherine and {Oregon Health Study Group}},
+  journal={The Quarterly Journal of Economics},
+  volume={127},
+  number={3},
+  pages={1057--1106},
+  year={2012},
+  doi={10.1093/qje/qjs020}
+}
+
+@article{kobrosly2020causalcurve,
+  title={causal-curve: A Python Causal Inference Package to Estimate Causal Dose-Response Curves},
+  author={Kobrosly, Roni W.},
+  journal={Journal of Open Source Software},
+  volume={5},
+  number={52},
+  pages={2523},
+  year={2020},
+  doi={10.21105/joss.02523}
+}
diff --git a/paper/paper.md b/paper/paper.md
@@ -0,0 +1,79 @@
+---
+title: 'dte_adj: A Python Package for Estimating Distributional Treatment Effects in Randomized Experiments'
+tags:
+  - Python
+  - randomized experiments
+  - causal inference
+  - distributional treatment effects
+  - machine learning
+  - variance reduction
+authors:
+  - name: Tomu Hirata
+    orcid: 0009-0006-3140-291X
+    corresponding: true
+    affiliation: "1, 3"
+  - name: Undral Byambadalai
+    affiliation: 1
+  - name: Tatsushi Oka
+    affiliation: "1, 2"
+  - name: Shota Yasui
+    affiliation: 1
+affiliations:
+ - name: CyberAgent, Inc., Japan
+   index: 1
+ - name: Keio University, Japan
+   index: 2
+ - name: Databricks, Inc., United States
+   index: 3
+date: 24 August 2025
+bibliography: paper.bib
+---
+
+# Summary
+
+`dte_adj` is a Python package for estimating distributional treatment effects (DTEs) in randomized experiments (RCTs, also known as A/B tests). Unlike traditional approaches that focus on average treatment effects, `dte_adj` enables researchers to analyze the full distributional impact of interventions across different outcome levels. The package implements machine learning-enhanced regression adjustment methods for variance reduction, supports multiple experimental designs including simple randomization, covariate-adaptive randomization, and settings with imperfect compliance, and provides a scikit-learn compatible API with comprehensive functionality for computing distribution functions, probability treatment effects, and quantile treatment effects with confidence intervals.
+
+# Statement of Need
+
+Randomized experiments have been fundamental to scientific inquiry since @fisher1935design, providing the gold standard for causal inference. While most experimental analyses focus on average treatment effects (ATEs), many research questions require understanding how treatments affect the entire distribution of outcomes. Distributional treatment effects (DTEs) capture these richer patterns, revealing heterogeneous impacts across different outcome levels that averages can mask. For example, a policy intervention might have no effect on average income while substantially reducing poverty rates at lower quantiles, or a medical treatment might benefit patients at the tails of the distribution differently than those near the median.
+
+Despite the growing importance of distributional analysis in economics, medicine, and technology, the Python ecosystem lacks comprehensive tools for DTE estimation with modern variance reduction techniques. Researchers often resort to basic empirical CDFs or manual implementations that lack statistical rigor. `dte_adj` fills this gap by providing a unified framework for distributional treatment effect analysis that integrates state-of-the-art machine learning methods for improved precision, rigorous confidence interval construction, and support for complex experimental designs.
+
+# State of the Field
+
+Several Python packages address causal inference, but none focus on distributional treatment effects with machine learning-based variance reduction:
+
+- **SciPy** [@2020SciPy-NMeth]: Provides basic empirical cumulative distribution functions but offers no functionality for treatment effect estimation or confidence interval construction in experimental settings.
+- **DoWhy** [@dowhy]: Focuses on causal graph-based inference and average treatment effects, without distributional analysis capabilities.
+- **EconML** [@econml]: Incorporates machine learning for heterogeneous treatment effect estimation (CATE) but does not address distributional effects.
+- **causal-curve** [@kobrosly2020causalcurve]: Estimates dose-response curves but targets continuous treatments rather than distributional outcomes.
+
+In the R ecosystem, packages like `qte` provide quantile treatment effect estimation but lack machine learning integration for variance reduction. `dte_adj` uniquely combines: (1) distributional treatment effect estimation across the full outcome distribution, (2) machine learning-enhanced regression adjustment for precision gains, and (3) support for multiple experimental designs including covariate-adaptive randomization and imperfect compliance settings.
+
+# Software Design
+
+`dte_adj` follows a class-based architecture with a template method pattern, where a base class defines the algorithm structure and subclasses implement design-specific computations:
+
+- **`SimpleDistributionEstimator`** and **`AdjustedDistributionEstimator`**: For simple randomized experiments, implementing methods from @byambadalai2024estimatingdistributionaltreatmenteffects.
+- **`SimpleStratifiedDistributionEstimator`** and **`AdjustedStratifiedDistributionEstimator`**: For covariate-adaptive randomization designs, implementing methods from @byambadalai2025efficientestimationdistributionaltreatment.
+- **`SimpleLocalDistributionEstimator`** and **`AdjustedLocalDistributionEstimator`**: For settings with imperfect compliance, implementing methods from @byambadalai2025imperfectcompliance.
+
+All estimators implement a consistent API with three primary methods: `predict_dte()` for distributional treatment effects, `predict_pte()` for probability treatment effects over intervals, and `predict_qte()` for quantile treatment effects. The adjusted estimators use K-fold cross-fitting to prevent overfitting and support both single-task and multi-task learning modes [@hirata2025efficientscalableestimationdistributional] for computational efficiency. Bootstrap methods provide confidence intervals with multiple variance estimation approaches.
+
+![Distributional treatment effects for the Hillstrom email marketing dataset [@hillstrom2008], comparing Women's vs Men's email campaigns. The simple estimator (left, purple) and ML-adjusted estimator (right, green) show that adjustment substantially tightens confidence bands, demonstrating the variance reduction benefit of regression adjustment.](hillstorm_dte.png)
+
+![Local distributional treatment effects for emergency department costs in the Oregon Health Insurance Experiment [@finkelstein2012], estimated using `SimpleLocalDistributionEstimator` (left) and `AdjustedLocalDistributionEstimator` (right). Health insurance coverage shifts the distribution of ED costs, with ML adjustment again yielding narrower confidence intervals.](oregon_ldte_costs_comparison.png)
+
+# Research Impact Statement
+
+The methods implemented in `dte_adj` have been published across machine learning and econometrics venues: ICML 2024 [@byambadalai2024estimatingdistributionaltreatmenteffects], Econometric Reviews [@oka2025regression], ICML 2025 [@byambadalai2025efficientestimationdistributionaltreatment], and NeurIPS 2025 [@byambadalai2025imperfectcompliance]. The package has been applied in industry settings, including analyzing the distributional impact of content promotion on user engagement at ABEMA, a major video streaming platform [@yasui2026abema]. The documentation includes tutorials demonstrating applications to the Hillstrom email marketing dataset (Figure 1) and the Oregon Health Insurance Experiment (Figure 2), facilitating adoption by researchers in economics, marketing, and healthcare.
+
+# AI Usage Disclosure
+
+Generative AI tools (Claude) were used to assist with documentation writing and code review during development. All AI-generated content was reviewed and validated by the human authors.
+
+# Acknowledgements
+
+We thank CyberAgent, Inc. for supporting this research and the open-source community for valuable feedback during development.
+
+# References
