From 2b8def101fc8d66eda934af395b274b3c3ae914a Mon Sep 17 00:00:00 2001 From: Vahid Ahmadi Date: Wed, 27 May 2026 11:15:11 +0200 Subject: [PATCH] Add fiscal-year uprating planning docs page (#1437 #1436) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #1437 is filed as a low-priority enhancement to align uprating indices with the UK fiscal year (April 6) rather than the current January 1 dates plus convert_to_fiscal_year_parameters post-process workaround. #1436 proposes the inverse — keep January 1 dates but resolve the conversion lazily at query time. This page captures both, the relationship between them, and the recommended sequencing (#1437 first, then re-evaluate #1436). It also identifies the four pipeline changes the April-date move needs: 1. yoy_growth.yaml dates from YYYY-01-01 to YYYY-04-06, 2. create_economic_assumption_indices.py emitting at April 6, 3. confirm policyengine-core's uprate_parameters doesn't hardcode January 1, 4. remove convert_to_fiscal_year_parameters and its call site in tax_benefit_system.py. The headline correctness gain is that mid-year queries (e.g. param("2026-05-15")) currently return the wrong fiscal-year value, and the planned change fixes that. --- changelog.d/1437.md | 1 + .../assumptions/fiscal-year-uprating-plan.md | 120 ++++++++++++++++++ 2 files changed, 121 insertions(+) create mode 100644 changelog.d/1437.md create mode 100644 docs/book/assumptions/fiscal-year-uprating-plan.md diff --git a/changelog.d/1437.md b/changelog.d/1437.md new file mode 100644 index 000000000..e53377dae --- /dev/null +++ b/changelog.d/1437.md @@ -0,0 +1 @@ +- Add a planning docs page at `docs/book/assumptions/fiscal-year-uprating-plan.md` describing the proposed move from January-1 to April-6 dates for OBR growth indices (#1437) and how it relates to the on-the-fly conversion alternative (#1436); identifies the four pipeline changes needed (yoy_growth.yaml dates, create_economic_assumption_indices.py, policyengine-core's `uprate_parameters` assumptions, and removing `convert_to_fiscal_year_parameters`) and notes the user-visible correctness gain at mid-year queries. diff --git a/docs/book/assumptions/fiscal-year-uprating-plan.md b/docs/book/assumptions/fiscal-year-uprating-plan.md new file mode 100644 index 000000000..3dd8bfa2e --- /dev/null +++ b/docs/book/assumptions/fiscal-year-uprating-plan.md @@ -0,0 +1,120 @@ +# Fiscal-year uprating: April-date alignment plan + +```{note} +**Planning page.** PolicyEngine UK currently stores OBR growth indices +at **January 1** dates and post-processes parameters to the fiscal year +via `convert_to_fiscal_year_parameters`. This page captures the proposed +move to **April 6** dates throughout, tracked under +[#1437](https://github.com/PolicyEngine/policyengine-uk/issues/1437), +and the on-the-fly alternative tracked under +[#1436](https://github.com/PolicyEngine/policyengine-uk/issues/1436). +This is a low-priority enhancement — the current approach is correct +for annual analysis after PR #1435 extended the fiscal-year-conversion +range out to 2040. +``` + +## Current pipeline + +The model currently does three things in sequence: + +1. **OBR indices** live at January 1 dates in + [`parameters/gov/economic_assumptions/yoy_growth.yaml`](../../../policyengine_uk/parameters/gov/economic_assumptions/yoy_growth.yaml) + (annual calendar-year growth rates) and the cumulative indices are + generated by + [`create_economic_assumption_indices.py`](../../../policyengine_uk/parameters/gov/economic_assumptions/create_economic_assumption_indices.py) + at January 1 dates (`f"{year}-01-01"`). +2. **`uprate_parameters`** in `policyengine-core` writes uprated + parameter values at the same dates as the uprating index, so all + uprated parameters land at January 1. +3. **`convert_to_fiscal_year_parameters`** in + [`utils/parameters.py`](../../../policyengine_uk/utils/parameters.py) + post-processes the parameter tree by sampling each parameter at + April 30 and overwriting the whole calendar year with that value. + This is what produces the fiscal-year-correct outputs PolicyEngine + ships today. + +This works for annual simulations but has three downsides: + +- **Intra-year queries are wrong.** `param("2026-03-15")` returns the + fiscal-year-2026 value when April 6 2026 hasn't happened yet; the + intended pre-April-6 value is the fiscal-year-2025 value. +- The fiscal-year conversion is a **post-process workaround**, not a + correct upstream model. +- Parameter `values_list` entries appear at January 1 dates even + though the legislative change usually takes effect on April 6. + +## Proposed change (#1437): store at April 6 throughout + +Update each step of the pipeline so it speaks fiscal years natively: + +1. Move the dates in `yoy_growth.yaml` from `YYYY-01-01` to + `YYYY-04-06` for every entry, ensuring the growth rate that applies + between fiscal year T and fiscal year T+1 sits at the start of T+1. +2. Update `create_economic_assumption_indices.py` to emit cumulative + indices at April 6 dates. +3. Confirm that `policyengine-core`'s `uprate_parameters` does not bake + any January-1 assumption — if it does, parametrise the date. +4. **Remove** `convert_to_fiscal_year_parameters` and its call site in + [`tax_benefit_system.py`](../../../policyengine_uk/tax_benefit_system.py). + Once steps 1-3 land, the natural answer at every date is already the + fiscal-year-correct value. + +### What changes for users + +| Query | Today | After April-date move | +|-------|-------|------------------------| +| `param("2026")` (year-string) | Fiscal year 2026/27 value (post-process) | Fiscal year 2026/27 value (direct) | +| `param("2026-03-15")` (pre-April 6) | Fiscal year 2025/26 value (correct) | Fiscal year 2025/26 value (correct) | +| `param("2026-05-15")` (post-April 6) | Fiscal year 2025/26 value (**wrong**) | Fiscal year 2026/27 value (correct) | +| `values_list` dates | January 1 | April 6 | + +The third row is the headline correctness gain. + +### Scope + +- Pure documentation/wiring change — **no household-impact values + should move** for any annual query. +- Regression tests must include both an annual-query and an + April-30-style mid-year-query case so the next refactor can't + reintroduce the off-by-three-months bug. +- Reform-side parameters (legislated rate changes) should already be + dated April 6 in their YAML files (most are); spot-check the + exceptions during the migration. + +## On-the-fly alternative (#1436) + +The complementary issue [#1436](https://github.com/PolicyEngine/policyengine-uk/issues/1436) +proposes the inverse: instead of moving every date to April 6, keep +indices at January 1 and resolve the fiscal-year conversion lazily at +query time. + +That would let the conversion stop being a 26-year pre-computation step +at startup (the `range(2015, 2041)` loop in +`convert_to_fiscal_year_parameters`) and remove the year-range +upper-bound altogether. + +The two approaches are mutually exclusive: either we change the data to +match the queries (#1437) or we change the resolver to translate +queries to the data (#1436). Recommendation: + +- Prefer **#1437 (April dates)** for correctness and ergonomics — the + underlying policy change happens on April 6, so the YAML should say + April 6. +- Consider **#1436 (on-the-fly)** if a refactor of + `convert_to_fiscal_year_parameters` becomes hot-path for any specific + user — it's primarily a startup-cost optimisation. + +## Sequencing + +1. Land #1437 first: it removes the post-process workaround and makes + the YAML files self-explanatory. +2. Once #1437 is live, evaluate whether the lazy approach in #1436 is + still worth doing — most of its benefit is already captured by step 4 + above ("remove `convert_to_fiscal_year_parameters`"). + +## References + +- Issue [#1437](https://github.com/PolicyEngine/policyengine-uk/issues/1437) — April-date alignment. +- Issue [#1436](https://github.com/PolicyEngine/policyengine-uk/issues/1436) — on-the-fly conversion exploration. +- PR [#1435](https://github.com/PolicyEngine/policyengine-uk/pull/1435) — extended the fiscal-year-conversion range to 2040. +- [`growthfactors.md`](./growthfactors.md) — the underlying OBR growth rates.