Add Arch-backed PE target parity adapters

Author: MaxGhenis

docs/arch-target-gap-queue.md

@@ -0,0 +1,104 @@
+# Arch Target Gap Queue
+
+The Arch target gap queue is a Microplex-side review tool. It compares a
+Microplex target profile to a queryable Arch target DB and emits rows that help
+humans or agents decide what Arch source work is missing.
+
+The queue does not make Arch own Microplex target selection. Profile membership,
+source aging, reconciliation, activation, and model-variable aliases remain in
+`microplex-us`.
+
+## Boundary Rules
+
+- Arch stores publisher/source facts with provenance, constraints, periods,
+  geography, and source lineage.
+- Arch should not duplicate a source fact only because Microplex names a model
+  variable differently.
+- Microplex adapters may map one Arch source fact into simulator-specific target
+  semantics. For example, Arch
+  `irs_soi.returns_with_income_tax_after_credits` can satisfy the
+  PolicyEngine `income_tax_positive` count target because SOI Table 1.1 reports
+  the count of returns with positive income tax after credits.
+- A gap row is an authoring hint, not proof that a source exists.
+- Rows marked as source-mapping review or deprioritized must be reviewed before
+  assigning loader work to agents.
+
+## Categories
+
+`gap_category` is the high-level agent-readiness taxonomy:
+
+| Category | Meaning | Default action |
+| --- | --- | --- |
+| `covered` | An Arch target record already satisfies the target cell. | No task. |
+| `ready_primary_loader` | The expected publisher source and Arch variable shape are known, but the record is missing. | Assign source-loader/spec work. |
+| `ready_rollup_or_geography` | The Arch variable exists but not at the requested geography. | Add rollup/geography records or review source geography. |
+| `adapter_or_constraint_review` | The Arch variable exists at the geography, but filters or adapter matching do not cover the cell. | Review constraints and adapter mapping. |
+| `source_mapping_review` | The queue cannot identify a defensible source fact or Arch variable shape. | Human source-mapping review first. |
+| `survey_or_model_input_deprioritized` | The cell is currently treated as a survey/model-input proxy rather than a primary administrative source task. | Defer unless a primary source is identified. |
+
+`loader_status` is the lower-level diagnostic used to derive the category. Use
+`gap_category` for agent routing and `loader_status` for debugging why a cell
+landed there.
+
+## Current PolicyEngine Broad Profile Boundary
+
+The current Arch-backed PE broad profile coverage intentionally stops before
+survey-heavy or model-input cells such as rent, net worth, child support,
+medical-premium subcomponents, SPM expenses, and `ssn_card_type`. Those rows are
+not ready for automated source-loader agents under the primary-source-first
+policy.
+
+## Current Local Snapshot
+
+Snapshot date: 2026-05-19.
+
+Inputs:
+
+- `/Users/maxghenis/CosilicoAI/arch/arch/fixtures/consumer_facts.jsonl`
+- `/Users/maxghenis/CosilicoAI/arch/macro/targets.db`
+
+Command:
+
+```bash
+uv run microplex-us-arch-target-refresh \
+  --artifact-root /Users/maxghenis/CosilicoAI/arch \
+  --period 2024 \
+  --profile pe_native_broad \
+  --output-dir artifacts/arch-target-coverage
+```
+
+Coverage:
+
+- 189 target cells in `pe_native_broad`
+- 138 covered
+- 51 uncovered
+- 73.0% coverage
+- national: 79 of 116 covered
+- state: 59 of 73 covered
+
+Gap categories:
+
+| Category | Rows |
+| --- | ---: |
+| `source_mapping_review` | 26 |
+| `survey_or_model_input_deprioritized` | 12 |
+| `adapter_or_constraint_review` | 10 |
+| `ready_rollup_or_geography` | 3 |
+
+Generated outputs:
+
+- `artifacts/arch-target-coverage/pe_native_broad_2024_coverage.json`
+- `artifacts/arch-target-coverage/pe_native_broad_2024_gaps.json`
+- `artifacts/arch-target-coverage/pe_native_broad_2024_gaps.csv`
+- `artifacts/arch-target-coverage/pe_native_broad_2024_summary.md`
+
+Remaining work is concentrated in:
+
+- source-mapping review for the newly expanded PE parity cells, especially
+  domains whose expected Arch concept is not yet encoded in the gap taxonomy
+- adapter or constraint review where Arch has the variable at the right
+  geography but the Microplex adapter does not yet match the PE target cell
+- a small rollup/geography queue for variables loaded in Arch but not at the
+  requested national or state target geography
+- survey/model-input proxy cells that remain deprioritized until a primary
+  publisher source is identified

pyproject.toml

@@ -13,7 +13,8 @@ authors = [
 ]
 requires-python = ">=3.13"
 dependencies = [
-    "microplex[calibrate]",
+    "microplex[calibrate]>=0.2.0",
+    "microunit>=0.1.0",
     "duckdb>=1.2",
     "requests>=2.31",
 ]
@@ -23,20 +24,34 @@ dev = [
     "pytest>=7.0",
     "ruff>=0.1",
 ]
+r2 = [
+    "boto3>=1.34",
+]
 policyengine = [
     "microimpute==1.15.1 ; python_full_version >= '3.12' and python_full_version < '3.15'",
     "policyengine-us==1.587.0; python_version >= '3.11' and python_version < '3.15'",
+    "spm-calculator>=0.3.1",
 ]
 
 [project.urls]
 Repository = "https://github.com/CosilicoAI/microplex-us"
 
 [project.scripts]
+microplex-us-arch-target-coverage = "microplex_us.targets.arch:main_coverage"
+microplex-us-arch-target-gaps = "microplex_us.targets.arch:main_gaps"
+microplex-us-arch-target-parity = "microplex_us.targets.arch:main_parity"
+microplex-us-arch-target-refresh = "microplex_us.targets.arch:main_refresh"
+microplex-us-arch-target-smoke = "microplex_us.targets.arch:main_smoke"
 microplex-us-build-aca-ptc-multipliers = "microplex_us.targets.aca_ptc:main"
 microplex-us-backfill-pe-native-audit = "microplex_us.pipelines.backfill_pe_native_audit:main"
 microplex-us-backfill-pe-native-scores = "microplex_us.pipelines.backfill_pe_native_scores:main"
 microplex-us-check-site-snapshot = "microplex_us.pipelines.check_site_snapshot:main"
+microplex-us-pe-dataset-readiness = "microplex_us.pipelines.pe_us_dataset_readiness:main"
+microplex-us-dashboard = "microplex_us.pipelines.dashboard:main"
+microplex-us-pe-native-calibration-benchmark = "microplex_us.pipelines.pe_native_calibration_benchmark:main"
 microplex-us-pe-native-target-diagnostics = "microplex_us.pipelines.pe_native_scores:main_target_diagnostics"
+microplex-us-r2-archive-artifact = "microplex_us.pipelines.r2_artifacts:main"
+microplex-us-reweight-cd-age-targets = "microplex_us.pipelines.cd_age_reweighting:main"
 microplex-us-score-pe-native-loss = "microplex_us.pipelines.pe_native_scores:main"
 microplex-us-version-bump-benchmark = "microplex_us.pipelines.version_benchmark:main"
 
@@ -69,3 +84,4 @@ ignore = [
 
 [tool.uv.sources]
 microplex = { path = "../microplex", editable = true }
+microunit = { path = "../microunit", editable = true }