Finish migration from yaml-changelog to towncrier (#795)

* Finish migration from yaml-changelog to towncrier; point CONTRIBUTING.md at shared guide

This repo was partway through the yaml-changelog → towncrier migration
already: `pr.yaml` ran `towncrier check`, `.github/bump_version.py`
inferred version bumps from `changelog.d/` fragment types, and
`pyproject.toml` carried `[tool.towncrier]` config. The stale bits
that remained:

- Unused dep `yaml-changelog>=0.1.7` in `pyproject.toml`
- Unused `.github/workflows/reusable_changelog_check.yaml` (not
  wired into any call-graph)
- Zero-byte `changelog_entry.yaml` at the repo root
- 601-line `changelog.yaml` whose entire contents are already
  compiled into `CHANGELOG.md`
- A CONTRIBUTING.md that described the old yaml-changelog flow and
  told contributors to edit `changelog_entry.yaml`

This PR drops all of the above and rewrites CONTRIBUTING.md to point
at the new shared PolicyEngine guide
(https://github.com/PolicyEngine/.github/blob/main/CONTRIBUTING.md,
proposed in PolicyEngine/.github#3) plus a repo-specific section on
commands, test placement, dataset-versioning rules, and anti-patterns.

After this lands the repo matches the rest of the org (uk-data, core,
uk, us, microdf, policyengine.py) on a single towncrier flow.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Update uv.lock after removing yaml-changelog

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: MaxGhenis

.github/CONTRIBUTING.md

@@ -1,11 +1,63 @@
-## Updating data
+# Contributing to policyengine-us-data
 
-If your changes present a non-bugfix change to one or more datasets which are cloud-hosted (CPS, ECPS and PUF), then please change both the filename and URL (in both the class definition file and in `storage/upload_completed_datasets.py`. This enables us to store historical versions of datasets separately and reproducibly.
+See the [shared PolicyEngine contribution guide](https://github.com/PolicyEngine/.github/blob/main/CONTRIBUTING.md) for cross-repo conventions (towncrier changelog fragments, `uv run`, PR description format, anti-patterns). This file covers policyengine-us-data specifics.
+
+## Commands
+
+```bash
+make install                 # install deps (uv)
+make format                  # format (required)
+make test-unit               # unit tests (synthetic / mocked, seconds)
+make test-integration        # integration tests (need built H5 datasets)
+make test                    # both
+make data                    # full dataset build (long)
+make push-pr-branch          # push to upstream with correct tracking (use before opening PRs)
+uv run pytest tests/unit/datasets/ -v
+```
+
+Python 3.12–3.14. Default branch: `main`.
+
+## Test organisation
+
+- `tests/unit/` — self-contained (synthetic data, mocks, checked-in fixtures). Run in seconds with no external deps.
+  - `unit/datasets/` — dataset code
+  - `unit/calibration/` — calibration code
+- `tests/integration/` — requires built H5 datasets, HuggingFace downloads, `Microsimulation` objects, or DB ETL. Named after the dataset under test (e.g. `test_cps.py` tests `cps_2024.h5`).
+
+**Placement rules:**
+
+- **Never** put tests that need H5 files or `Microsimulation` in `unit/`.
+- **Never** put synthetic-only tests in `integration/`.
+- Sanity checks (value ranges, population counts) go in the per-dataset integration file, not a separate sanity file.
+- When adding an integration test, extend the existing per-dataset file if one exists.
+
+## Updating datasets
+
+If your change is a non-bugfix update to a cloud-hosted dataset (CPS, enhanced CPS, PUF), bump both the filename and URL in the class definition and in `storage/upload_completed_datasets.py`. That lets us store historical dataset versions separately and reproducibly.
 
 ## Opening PRs
 
-Push PR branches to the upstream `PolicyEngine/policyengine-us-data` repository, not to a personal fork. From the repo root, run:
+**Always create branches on the upstream repo, not a fork.** Fork PRs can't access workflow secrets and will fail on data-download steps. The convenience target:
+
+```bash
+make push-pr-branch
+```
+
+pushes the current branch to `upstream` with the correct tracking so `gh pr create` just works.
+
+## Repo-specific anti-patterns
+
+- **Never fabricate data or results.** This is a research codebase; reproducible aggregates only. Use `[TO BE CALCULATED]` placeholders if a number isn't computed yet.
+- **Don't** open PRs from personal forks (CI will fail on secrets).
+- **Don't** add `[codex]` or other agent-label prefixes to PR titles.
+- **Don't** skip full-build CI when touching the imputation or calibration pipeline.
+- **Don't** commit large binary artefacts — HuggingFace storage only.
+
+## CI workflows
 
-`make push-pr-branch`
+Five workflow files in `.github/workflows/`:
 
-This avoids the fork-only CI failure path and sets the upstream tracking branch correctly before opening the PR.
+- `pr.yaml` — fork check, lint, uv.lock freshness, towncrier fragment check, unit tests, smoke test, docs build. Integration tests trigger when files in `policyengine_us_data/`, `modal_app/`, or `tests/integration/` change. ~2–3 min for the unit path.
+- `push.yaml` — on push to main: either version-bump + PyPI publish (on `Update package version` commits), or a full Modal data build with integration tests (on everything else).
+- `pipeline.yaml` — dispatch only, spawns the H5 generation pipeline on Modal with configurable GPU/epochs/workers.
+- `local_area_publish.yaml` / `local_area_promote.yaml` — manual dispatch to build/stage and then promote local-area H5 files.

changelog.d/migrate-to-towncrier.changed.md

@@ -0,0 +1 @@
+Migrate the changelog tooling from `yaml-changelog` (`changelog_entry.yaml` + `changelog.yaml` + `build-changelog`) to towncrier (`changelog.d/<branch>.<type>.md` fragments). The repo's CI already ran `towncrier check` in `pr.yaml` and `bump_version.py` already read fragments from `changelog.d/`; this drops the leftover yaml-changelog artefacts (unused dep, unused reusable workflow, zero-byte `changelog_entry.yaml`, and duplicated `changelog.yaml` whose contents are already in `CHANGELOG.md`) so the tooling story matches the rest of the org.