Merge pull request #711 from PlanExeOrg/feat/napkin-math-validate-parameters-python

feat(napkin-math): deterministic Python validator for parameters.json

Author: neoneye

experiments/napkin_math/.claude/skills/extract-parameters-from-digest/system-prompt.txt

@@ -221,6 +221,17 @@ If you would naturally write "x_gap = expected - capacity" with threshold "<= 0"
 
 Apply this rule even when the underlying source phrasing is "the gap is X". The source text is allowed to use "gap"; the calculation output id is not.
 
+Contractual-gate naming:
+
+When a calculated window, margin, or trigger represents a contractual gate enforced by a specific counterparty (sponsor, lender, regulator, agency, court, investor, grantor, prime contractor), prefix the id with the counterparty so the gate's contractual — rather than operational — nature is visible to downstream consumers. Examples:
+
+- sponsor_profitability_trigger_window_days  (not "effective_profitability_window_days")
+- lender_dscr_covenant_margin                (not "dscr_margin")
+- regulator_emissions_threshold_margin       (not "emissions_margin")
+- prime_contractor_milestone_window_days     (not "milestone_window_days")
+
+The prefix tells the next reader that failing this gate is a contract-validity failure, not an operational shortfall — different remediation paths apply. When the source plan does not name a specific counterparty, use the neutral non-prefixed form.
+
 Derive coupled stressors instead of sampling them independently:
 
 If a stressor variable (a shortfall, deficit, overrun, drain, leakage) is mechanically derivable from quantities the model already covers, declare it as a calculation. Do not put it in missing_values_to_estimate as if it were an independent input. Sampling a derived stressor independently:
@@ -835,6 +846,32 @@ When a formula requires a missing input, include that input in missing_values_to
 
 Avoid formula_hint variables that are semantically different from the extracted id. For example, do not use registered_population if the extracted value is target_population unless both are declared.
 
+Unmodelled existential gates (optional but recommended):
+
+Some plans depend on gates the deterministic Python model cannot evaluate — legal/political authorization, regulatory approval, compliance infrastructure (AML/KYC, certifications, licences), an external actor's binding commitment treated as a fixed input. These gates have no quantifiable threshold the Monte Carlo can test, but their failure would shut the plan down independently of any financial or operational threshold the model evaluates.
+
+When the source digest (typically the Premortem, Expert Criticism, Risks, or Selected Scenario sections) names such a gate, declare it in `unmodelled_gates`. This tells downstream consumers that the simulation is a partial feasibility assessment, not a full one.
+
+Hard limit: at most 5 unmodelled_gates.
+
+For each unmodelled gate, return:
+- id: stable snake_case identifier ending in `_gate`
+- label: short human-readable name
+- why_it_matters: one or two sentences explaining the gate's role
+- source_anchor: which source section the gate is named in (one of: executive_summary, project_plan, selected_scenario, assumptions, review_plan, premortem, expert_criticism, data_collection)
+- consequence_if_false: what happens to the plan if this gate fails
+
+When to populate unmodelled_gates:
+- The plan depends on regulatory approval or post-facto authorization the model does not test.
+- The plan requires political acceptance, legitimacy, non-reversal, or social licence that no input represents.
+- The plan needs compliance infrastructure (AML/KYC banking partners, safety certifications, operating licences) the model treats as given.
+- The plan depends on an external actor's binding commitment (a grid operator, banking consortium, agency, court) that the model treats as a fixed input rather than a probabilistic gate.
+
+When to leave it empty (omit the field or use `[]`):
+- Every viability claim the source plan makes can be expressed as a formula tested against a quantifiable threshold.
+
+Do not use unmodelled_gates as a dumping ground for risks. Only include gates whose failure would end the plan independently of the financial or operational thresholds the model tests.
+
 Return this exact JSON shape:
 
 {
@@ -892,5 +929,14 @@ Return this exact JSON shape:
       "depends_on": [],
       "why_first": ""
     }
+  ],
+  "unmodelled_gates": [
+    {
+      "id": "",
+      "label": "",
+      "why_it_matters": "",
+      "source_anchor": "",
+      "consequence_if_false": ""
+    }
   ]
 }

experiments/napkin_math/.claude/skills/extract-parameters-from-full/system-prompt.txt

@@ -184,6 +184,17 @@ If you would naturally write "x_gap = expected - capacity" with threshold "<= 0"
 
 Apply this rule even when the underlying source phrasing is "the gap is X". The source text is allowed to use "gap"; the calculation output id is not.
 
+Contractual-gate naming:
+
+When a calculated window, margin, or trigger represents a contractual gate enforced by a specific counterparty (sponsor, lender, regulator, agency, court, investor, grantor, prime contractor), prefix the id with the counterparty so the gate's contractual — rather than operational — nature is visible to downstream consumers. Examples:
+
+- sponsor_profitability_trigger_window_days  (not "effective_profitability_window_days")
+- lender_dscr_covenant_margin                (not "dscr_margin")
+- regulator_emissions_threshold_margin       (not "emissions_margin")
+- prime_contractor_milestone_window_days     (not "milestone_window_days")
+
+The prefix tells the next reader that failing this gate is a contract-validity failure, not an operational shortfall — different remediation paths apply. When the source plan does not name a specific counterparty, use the neutral non-prefixed form.
+
 Derive coupled stressors instead of sampling them independently:
 
 If a stressor variable (a shortfall, deficit, overrun, drain, leakage) is mechanically derivable from quantities the model already covers, declare it as a calculation. Do not put it in missing_values_to_estimate as if it were an independent input. Sampling a derived stressor independently:
@@ -794,6 +805,32 @@ When a formula requires a missing input, include that input in missing_values_to
 
 Avoid formula_hint variables that are semantically different from the extracted id. For example, do not use registered_population if the extracted value is target_population unless both are declared.
 
+Unmodelled existential gates (optional but recommended):
+
+Some plans depend on gates the deterministic Python model cannot evaluate — legal/political authorization, regulatory approval, compliance infrastructure (AML/KYC, certifications, licences), an external actor's binding commitment treated as a fixed input. These gates have no quantifiable threshold the Monte Carlo can test, but their failure would shut the plan down independently of any financial or operational threshold the model evaluates.
+
+When the source report names such a gate (typically in risk register, executive summary, or expert review sections), declare it in `unmodelled_gates`. This tells downstream consumers that the simulation is a partial feasibility assessment, not a full one.
+
+Hard limit: at most 5 unmodelled_gates.
+
+For each unmodelled gate, return:
+- id: stable snake_case identifier ending in `_gate`
+- label: short human-readable name
+- why_it_matters: one or two sentences explaining the gate's role
+- source_anchor: which source section the gate is named in (one of: executive_summary, project_plan, selected_scenario, assumptions, review_plan, premortem, expert_criticism, data_collection)
+- consequence_if_false: what happens to the plan if this gate fails
+
+When to populate unmodelled_gates:
+- The plan depends on regulatory approval or post-facto authorization the model does not test.
+- The plan requires political acceptance, legitimacy, non-reversal, or social licence that no input represents.
+- The plan needs compliance infrastructure (AML/KYC banking partners, safety certifications, operating licences) the model treats as given.
+- The plan depends on an external actor's binding commitment (a grid operator, banking consortium, agency, court) that the model treats as a fixed input rather than a probabilistic gate.
+
+When to leave it empty (omit the field or use `[]`):
+- Every viability claim the source plan makes can be expressed as a formula tested against a quantifiable threshold.
+
+Do not use unmodelled_gates as a dumping ground for risks. Only include gates whose failure would end the plan independently of the financial or operational thresholds the model tests.
+
 Return this exact JSON shape:
 
 {
@@ -851,5 +888,14 @@ Return this exact JSON shape:
       "depends_on": [],
       "why_first": ""
     }
+  ],
+  "unmodelled_gates": [
+    {
+      "id": "",
+      "label": "",
+      "why_it_matters": "",
+      "source_anchor": "",
+      "consequence_if_false": ""
+    }
   ]
 }

experiments/napkin_math/.claude/skills/summarize-assessment/SKILL.md

@@ -92,11 +92,12 @@ Order is deliberate. Stable section names — programmatic consumers retrieve by
 
 - **`# Assessment: <plan name>`** — title plus a 2-line frontmatter (type, primary goal).
 - **`## Artifact contract`** — declares what this file is (an interpretation layer over the simulation artifacts) and what it is not (a copy of the raw simulation data, an external feasibility proof, a probability calibration).
-- **`## Machine summary`** — a JSON code block with the compact manifest: `assessment_schema_version` (currently `4`), `artifact_type`, `plan_name`, `artifact_set` (`version` / `plan_slug` / `relative_dir` — the portable identifier), `source_plan_dir` (absolute path; local-only), `primary_model_result` (a structured object: `overall_risk_band` ∈ doom/fragile/marginal/viable/unknown, `basis` — a one-line disclaimer that the band reflects the worst declared gate's pass-rate band and is not a calibrated whole-plan probability, `reason`, `worst_gate`, `worst_gate_pass_rate`), `validation_status`, `simulation` (n_runs/seed/distribution_default), `primary_failed_gates`, `primary_uncertainty_drivers`, `do_not_treat_as`, `schema_notes` (allowed enums for `overall_risk_band`, `verdict`, `basis`, `threshold_basis`, plus the `primary_model_result_semantics` disclaimer). The `basis_enum` is intentionally wider than what the current pipeline emits (`report_derived`, `model_assumption`) — it reserves `report_explicit`, `report_inferred`, `external_reference`, `manual_override`, `unknown` for future provenance types. JSON, not YAML — that is intentional.
+- **`## Machine summary`** — a JSON code block with the compact manifest: `assessment_schema_version` (currently `6`), `artifact_type`, `plan_name`, `artifact_set` (`version` / `plan_slug` / `relative_dir` — the portable identifier), `source_plan_dir` (absolute path; local-only), `primary_model_result` (a structured object: `overall_risk_band` ∈ doom/fragile/marginal/viable/unknown, `basis` — a one-line disclaimer that the band reflects the worst declared gate's pass-rate band and is not a calibrated whole-plan probability, `reason`, `worst_gate`, `worst_gate_pass_rate`), `validation_status`, `simulation` (n_runs/seed/distribution_default), `primary_failed_gates`, `primary_uncertainty_drivers`, `known_unmodelled_existential_gates` (flat list of ids the extractor flagged as existential but unmodelable; empty when none), `assessment_scope_warning` (string when the unmodelled list is non-empty, null when empty — a one-line warning to programmatic consumers that the simulation is partial), `do_not_treat_as`, `schema_notes` (allowed enums for `overall_risk_band`, `verdict`, `basis`, `threshold_basis`, plus the `primary_model_result_semantics` disclaimer). The `basis_enum` is intentionally wider than what the current pipeline emits (`report_derived`, `model_assumption`) — it reserves `report_explicit`, `report_inferred`, `external_reference`, `manual_override`, `unknown` for future provenance types. JSON, not YAML — that is intentional.
 - **`## Provenance map`** — table listing every intermediary file with its role and "open when" guidance. The first row points at `extract_parameters_input.md`, then parameters/bounds/calculations/scenarios/scenario_outputs/montecarlo_settings/montecarlo/validation.
-- **`## Modelling frame`** — the source plan's own statement of what the model is testing, lifted verbatim from `parameters.plan_summary.modelling_frame`.
+- **`## Modelling frame`** — the source plan's own statement of what the model is testing, lifted verbatim from `parameters.plan_summary.modelling_frame`. When `parameters.unmodelled_gates` is non-empty, this section gains a bold **Note** caveat naming the count of unmodelled existential gates and telling the reader the gate-verdict pass rates are conditional on those gates holding.
+- **`## Known unmodelled existential gates`** — table of gates the extractor flagged as existential to the plan but unmodelable by deterministic Python (legal authorization, political reversal, AML/banking compliance, external-actor commitments). Columns: Gate, Why it matters, Source anchor (which source section names it), Consequence if false. Section omitted entirely when `parameters.unmodelled_gates` is empty or absent.
 - **`## Simulation settings`** — n_runs, seed, distribution_default, validation status.
-- **`## Critical findings`** — bullets in severity order: DOOM gates, FRAGILE gates, scenario warnings, numbers the model could not compute (≥5% blank runs), still-missing inputs. Section omitted entirely when nothing qualifies.
+- **`## Critical findings`** — bullets in severity order. When `parameters.unmodelled_gates` is non-empty, a **SCOPE WARNING** bullet leads the section naming the unmodelled gate labels and pointing at the dedicated section. Then: DOOM gates, FRAGILE gates, scenario warnings, numbers the model could not compute (≥5% blank runs), still-missing inputs. Section omitted entirely when nothing qualifies.
 - **`## Gate verdicts`** — every declared threshold, worst-first, with the `min` marker on aggregate gates. Columns: marker, output, condition, **threshold basis** (`report_explicit` / `report_inferred` / `model_defined` / `unknown` — derived from the corresponding key_value's `value_type`), pass rate, verdict, meaning. Includes an `### Aggregation warning` sub-section when the thresholds use incompatible units and the plan declares no `min()` aggregate.
 - **`## Decision implications`** — one row per gate with verdict in DOOM/FRAGILE/MARGINAL. Five columns: Gate, Verdict, **Planning consequence** (templated by verdict), **Structural lever** (the top driver from `quartile_analysis` with the direction implied by its sign of Δ-pp), **Gate meaning** (the gate's own rationale lifted from `parameters.recommended_first_calculations[].why_first` or `derived_questions[].why_it_matters`, plus the threshold parameter the formula tests against). The Gate-meaning column surfaces plan-specific framing without inventing tactical advice; concrete revisions should be derived by reading the source report and the relevant intermediary artifacts.
 - **`## Failure drivers`** — one row per failing gate (DOOM or FRAGILE): top driver from `quartile_analysis` (max abs Δ-pp) and the conditional input restriction from `required_input_thresholds` that would lift the gate to 80%. Binding-gate frequencies for aggregates appear as bullets below the table.