Skip to content

Refine SSI disability training filters and targets#1131

Merged
MaxGhenis merged 7 commits into
mainfrom
codex/sipp-ssi-sga-income-filter
May 26, 2026
Merged

Refine SSI disability training filters and targets#1131
MaxGhenis merged 7 commits into
mainfrom
codex/sipp-ssi-sga-income-filter

Conversation

@MaxGhenis
Copy link
Copy Markdown
Contributor

@MaxGhenis MaxGhenis commented May 24, 2026

Summary

  • add an SGA screen to the SIPP SSI disability training candidate filter for nonblind records
  • replace the rough 2 * FBR total-income proxy with an approximate SSI countable-income screen using PE-US SSI exclusions
  • use SIPP job earnings for the disability model earned-income predictor without adding any manual model-version suffix
  • bump the pinned policyengine-us patch release to keep the data build dependency current
  • remove the legacy/new calibration dependency on ssi_federal_fiscal_year_outlays and train on ssi directly
  • use SSA actual 2024 Federal SSI annual payments ($59.665127B) as the 2024 SSI dollar target, falling back to CBO projections only when an SSA actual is unavailable
  • document why the SSA ASR annual target stays on annual ssi instead of OACT fiscal-year/payment-date month summing

Context

The SIPP SSI disability training frame should avoid treating records that are structurally unlikely to receive SSI under current law as clean non-disabled labels. This tightens the training candidate screen for earnings and income without changing the exported variable schema.

The SSI fiscal-year outlay formula is being removed from PolicyEngine-US. The us-data calibration surfaces now stop referencing that formula variable; explicit fiscal-year/payment-date timing logic belongs in the PE-US helper introduced in PolicyEngine/policyengine-us#8428.

For 2024, SSA has published actual Federal SSI total annual payments in SSI Annual Statistical Report 2024 Table 2. That ASR target allocates payments to the month due, so it aligns with annual ssi over January-December benefit months. It is not the same thing as OACT payment-date/fiscal accounting, so the calibration matrix remains the annual ssi calculation rather than summing a special payment-date month set.

The OACT payment-date comparison is useful as a guardrail: OACT Table IV.C1 reports CY2024 all Federal SSI payments of $63.080B and Table IV.C2 reports FY2024 all Federal SSI payments of $57.600B. That OACT CY-to-FY gap is $5.480B, about 8.7% of OACT CY2024, which is consistent with the 11-payment FY2024 issue plus lower benefit levels in the FY2024 months drawn from 2023. The much smaller ASR CY2024-to-OACT FY2024 gap reflects mixed bases: OACT obligations are not reduced for certain recovered overpayments remitted directly to Treasury, while ASR nets those recoveries out.

Related to #1130.

Tests

  • uv run ruff format policyengine_us_data/datasets/sipp/sipp.py tests/unit/datasets/test_sipp_ssi_disability.py
  • uv run ruff check policyengine_us_data/datasets/sipp/sipp.py tests/unit/datasets/test_sipp_ssi_disability.py
  • uv run pytest tests/unit/datasets/test_sipp_ssi_disability.py -q
  • uv run pytest tests/unit/calibration/test_loss_targets.py tests/unit/calibration/test_unified_matrix_builder.py tests/unit/calibration/test_target_config.py tests/unit/test_etl_national_targets.py tests/unit/calibration/test_hourly_wage_income_consistency.py tests/unit/calibration/test_compare_calibration_runs.py tests/unit/calibration/test_check_staging_sums.py tests/unit/calibration/test_validate_national_h5.py -q
  • uv run ruff check policyengine_us_data/utils/ssi_targets.py policyengine_us_data/utils/loss.py policyengine_us_data/calibration/unified_matrix_builder.py policyengine_us_data/calibration/chunked_matrix_assembler.py tests/unit/calibration/test_loss_targets.py tests/unit/calibration/test_unified_matrix_builder.py tests/unit/test_etl_national_targets.py
  • uv run ruff format --check policyengine_us_data/utils/ssi_targets.py policyengine_us_data/utils/loss.py policyengine_us_data/calibration/unified_matrix_builder.py policyengine_us_data/calibration/chunked_matrix_assembler.py tests/unit/calibration/test_loss_targets.py tests/unit/calibration/test_unified_matrix_builder.py tests/unit/test_etl_national_targets.py
  • uv run ruff check policyengine_us_data/utils/ssi_targets.py tests/unit/test_etl_national_targets.py
  • uv run ruff format --check policyengine_us_data/utils/ssi_targets.py tests/unit/test_etl_national_targets.py
  • uv run pytest tests/unit/test_etl_national_targets.py::test_extract_national_targets_uses_ssi_ssa_actual_when_available -q

@MaxGhenis MaxGhenis force-pushed the codex/sipp-ssi-sga-income-filter branch from 97ee509 to b60ff1b Compare May 24, 2026 15:53
@MaxGhenis MaxGhenis changed the title Refine SIPP SSI disability training filters Refine SSI disability training filters and targets May 24, 2026
@MaxGhenis MaxGhenis force-pushed the codex/sipp-ssi-sga-income-filter branch 3 times, most recently from bc791f7 to c884da6 Compare May 24, 2026 19:00
@MaxGhenis MaxGhenis marked this pull request as ready for review May 24, 2026 19:38
@MaxGhenis MaxGhenis force-pushed the codex/sipp-ssi-sga-income-filter branch from c884da6 to 366d8ab Compare May 25, 2026 03:27
@MaxGhenis MaxGhenis merged commit 9980b41 into main May 26, 2026
14 checks passed
@MaxGhenis MaxGhenis deleted the codex/sipp-ssi-sga-income-filter branch May 26, 2026 10:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant