[codex] Add policy decision explainability guide

stacknil · web-flow · commit 13fa0b60ce37 · 2026-05-09T13:25:16.000+08:00
Docs/example update adding policy decision explainability guidance, syncing reviewer-facing v0.7.0 references, and preserving deferred production PyPI status. No runtime, workflow, package metadata, tag, release, or publishing changes.
diff --git a/tools/sbom-diff-and-risk/README.md b/tools/sbom-diff-and-risk/README.md
@@ -10,11 +10,25 @@ It uses conservative heuristics for change intelligence. By default it does not
 
 This project has two different provenance stories:
 
-For a concise reviewer-facing overview, start with [docs/reviewer-brief.md](docs/reviewer-brief.md). For reproducible review evidence and verification commands, use [docs/reviewer-evidence-pack.md](docs/reviewer-evidence-pack.md). For machine-readable JSON output shape, see [docs/report-schema.md](docs/report-schema.md). For CI consumption of summary-only output, see [docs/summary-json-ci-cookbook.md](docs/summary-json-ci-cookbook.md).
-For a consumer-facing GitHub Actions example, see [docs/github-actions-consumer-example.md](docs/github-actions-consumer-example.md).
+For a concise reviewer-facing overview, start with
+[docs/reviewer-brief.md](docs/reviewer-brief.md). For reproducible review
+evidence and verification commands, use
+[docs/reviewer-evidence-pack.md](docs/reviewer-evidence-pack.md). For
+machine-readable JSON output shape, see
+[docs/report-schema.md](docs/report-schema.md). For policy decision
+explainability fields, see
+[docs/policy-decision-explainability.md](docs/policy-decision-explainability.md).
+For CI consumption of summary-only output, see
+[docs/summary-json-ci-cookbook.md](docs/summary-json-ci-cookbook.md).
+For a consumer-facing GitHub Actions example, see
+[docs/github-actions-consumer-example.md](docs/github-actions-consumer-example.md).
 
-1. If you want to verify `sbom-diff-and-risk` itself, start with [docs/verification.md](docs/verification.md).
-2. If you want to use `sbom-diff-and-risk` to analyze third-party dependency provenance, start with [Dependency provenance analysis](#dependency-provenance-analysis-opt-in) and [Dependency provenance reporting](#dependency-provenance-reporting).
+1. If you want to verify `sbom-diff-and-risk` itself, start with
+   [docs/verification.md](docs/verification.md).
+2. If you want to use `sbom-diff-and-risk` to analyze third-party dependency
+   provenance, start with
+   [Dependency provenance analysis](#dependency-provenance-analysis-opt-in)
+   and [Dependency provenance reporting](#dependency-provenance-reporting).
 
 ## Scope
 
diff --git a/tools/sbom-diff-and-risk/docs/github-actions-consumer-example.md b/tools/sbom-diff-and-risk/docs/github-actions-consumer-example.md
@@ -50,15 +50,15 @@ jobs:
           GH_TOKEN: ${{ github.token }}
         run: |
           mkdir -p .tooling/sbom-diff-risk
-          gh release download v0.6.0 \
+          gh release download v0.7.0 \
             --repo stacknil/scientific-computing-toolkit \
-            --pattern "sbom_diff_and_risk-0.6.0-py3-none-any.whl" \
+            --pattern "sbom_diff_and_risk-0.7.0-py3-none-any.whl" \
             --dir .tooling/sbom-diff-risk
 
       - name: Install sbom-diff-risk
         run: |
           python -m pip install \
-            .tooling/sbom-diff-risk/sbom_diff_and_risk-0.6.0-py3-none-any.whl
+            .tooling/sbom-diff-risk/sbom_diff_and_risk-0.7.0-py3-none-any.whl
 
       - name: Compare dependency evidence
         run: |
diff --git a/tools/sbom-diff-and-risk/docs/policy-decision-explainability.md b/tools/sbom-diff-and-risk/docs/policy-decision-explainability.md
@@ -0,0 +1,103 @@
+# Policy decision explainability
+
+This page explains the machine-readable policy decision metadata emitted in
+JSON reports. It is intended for reviewers and CI consumers who need to
+understand why a local policy rule produced a block, warning, or suppression.
+
+The fields described here are explainability metadata for local policy
+decisions. They are not dependency safety verdicts, CVE results, or proof that a
+package is safe or unsafe.
+
+## Where the fields appear
+
+Policy decision explanation fields appear only on policy finding objects, such
+as:
+
+- `policy_evaluation.blocking_violations`
+- `policy_evaluation.warning_violations`
+- `policy_evaluation.suppressed_violations`
+- `blocking_findings`
+- `warning_findings`
+- `suppressed_findings`
+- provenance policy impact sections
+
+Risk findings in `risks` remain the analyzer's local heuristic findings. They
+do not receive policy-decision metadata unless policy evaluation maps them into
+policy findings.
+
+## Field contract
+
+- `decision_reason`: Stable reason code for the policy decision.
+- `policy_rule`: Policy rule id that produced the decision.
+- `severity_source`: Source of the active severity, such as `block_on`,
+  `warn_on`, `default_block`, or `default_warn`; `null` when there is no active
+  severity.
+- `matched_threshold`: Configured threshold or allowlist value involved in the
+  decision, when applicable.
+- `observed_value`: Observed local value that was compared to the policy rule,
+  when applicable.
+
+The full JSON report shape is documented in [report-schema.md](report-schema.md).
+Policy configuration fields and supported rules are documented in
+[policy-schema.md](policy-schema.md).
+
+## Example interpretations
+
+A policy finding with:
+
+```json
+{
+  "decision_reason": "added_package_count_exceeded_threshold",
+  "policy_rule": "max_added_packages",
+  "severity_source": "block_on",
+  "matched_threshold": 0,
+  "observed_value": 1
+}
+```
+
+means the local policy compared an observed added-package count of `1` against a
+configured threshold of `0`, and the matching rule was active through
+`block_on`.
+
+A policy finding with:
+
+```json
+{
+  "decision_reason": "risk_finding_matched_policy_rule",
+  "policy_rule": "new_package",
+  "severity_source": "warn_on",
+  "matched_threshold": null,
+  "observed_value": "new_package"
+}
+```
+
+means a local heuristic risk finding matched the `new_package` policy rule, and
+the matching rule was active through `warn_on`.
+
+## CI and review usage
+
+Consumers can use these fields to group policy findings by rule, explain why a
+local gate failed, or build small job summaries. For example, a CI step can read
+`blocking_findings`, print each `policy_rule` and `decision_reason`, and fail
+only because the tool already returned a policy failure exit code.
+
+Use `summary.policy` for compact counts and status. Use policy finding
+explanation fields when a reviewer needs to inspect why the status was
+`warn` or `fail`.
+
+## Compatibility notes
+
+- The fields are additive JSON metadata for policy findings.
+- `summary.policy` is unchanged and remains the compact count/status surface.
+- Absence of policy findings means policy evaluation did not produce findings
+  for that section.
+- Absence of policy explanation fields outside policy finding objects is
+  expected.
+- Consumers should treat unrecognized future fields as additive report data.
+
+## Non-claims
+
+- The fields do not resolve CVEs.
+- The fields do not claim a package is safe or unsafe.
+- The fields do not add network behavior.
+- The fields do not replace human review of local policy choices.
diff --git a/tools/sbom-diff-and-risk/docs/policy-schema.md b/tools/sbom-diff-and-risk/docs/policy-schema.md
@@ -106,6 +106,10 @@ Version `3` supports every version `1` and `2` rule id plus:
 - Scorecard evidence remains an auxiliary trust signal. A high score is not
   proof of safety, and missing Scorecard data is not proof of risk.
 
+For the JSON policy finding explanation fields emitted after policy evaluation,
+see
+[policy-decision-explainability.md](policy-decision-explainability.md).
+
 ## Version 1 example
 
 ```yaml
diff --git a/tools/sbom-diff-and-risk/docs/pypi-production-publishing-decision.md b/tools/sbom-diff-and-risk/docs/pypi-production-publishing-decision.md
@@ -4,14 +4,21 @@ This page records the PR 5 production PyPI gate for `sbom-diff-and-risk`.
 
 ## PR 5 decision
 
-Production PyPI publishing is **deferred, but conditionally allowed after the prerequisites below are complete**. In short: production PyPI is currently deferred.
-
-PR 5 does not add an enabled production publishing workflow and does not publish to production PyPI. The successful TestPyPI Trusted Publishing dry-run proves that the package metadata can render on TestPyPI and that the TestPyPI OIDC path can work, but it is not automatic proof that production PyPI publishing is ready.
+Production PyPI publishing is **deferred, but conditionally allowed after the
+prerequisites below are complete**. In short: production PyPI is currently
+deferred.
+
+PR 5 does not add an enabled production publishing workflow and does not publish
+to production PyPI. The successful TestPyPI Trusted Publishing dry-run proves
+that the package metadata can render on TestPyPI and that the TestPyPI OIDC path
+can work, but it is not automatic proof that production PyPI publishing is
+ready.
 
 The production gate is intentionally conservative because:
 
 - the production PyPI project does not currently exist under the intended name
-- the package metadata has moved to `0.5.0` for the GitHub Release, but production PyPI publishing has not been enabled
+- the package metadata has moved beyond the TestPyPI dry-run version, but
+  production PyPI publishing has not been enabled
 - the first production upload should be a deliberate release version, not an old dry-run version
 - the production PyPI pending publisher or trusted publisher has not been configured
 - the production GitHub environment has not yet been confirmed
@@ -26,18 +33,26 @@ As checked on April 26, 2026:
 - `https://test.pypi.org/pypi/sbom-diff-and-risk/json` returned `200`
 - TestPyPI reports `sbom-diff-and-risk` version `0.4.1`
 
-This means the intended production project name is not currently visible on production PyPI, while the TestPyPI dry-run project exists. Treat the production name as available for this decision, but re-check immediately before configuration because PyPI can reserve, prohibit, or receive new projects at any time. The first production upload should use a production PyPI pending publisher unless the project is created by a maintainer before the publishing workflow is enabled.
+This means the intended production project name is not currently visible on
+production PyPI, while the TestPyPI dry-run project exists. Treat the production
+name as available for this decision, but re-check immediately before
+configuration because PyPI can reserve, prohibit, or receive new projects at any
+time. The first production upload should use a production PyPI pending publisher
+unless the project is created by a maintainer before the publishing workflow is
+enabled.
 
 ## First production version
 
 Do not publish `0.4.1` to production PyPI casually.
 
-The first production PyPI version should be `0.5.0` only if v0.5 is approved as the first production package release. Otherwise, defer to a later GitHub release tag.
+The first production PyPI version should be an explicitly approved current or
+future GitHub release tag. Do not backfill an older release casually.
 
 For the first production upload:
 
 - the GitHub tag should be `v<version>`
-- `tools/sbom-diff-and-risk/pyproject.toml` should declare the matching `<version>`
+- `tools/sbom-diff-and-risk/pyproject.toml` should declare the matching
+  `<version>`
 - the GitHub release and release assets should be available for the same tag
 - the production PyPI workflow should run from the matching tag ref
 - the production PyPI upload should use the checked distributions from that workflow run
@@ -55,23 +70,33 @@ Configure the production PyPI publisher to match this identity exactly:
 | Trusted Publisher workflow name field | `sbom-diff-and-risk-pypi.yml` |
 | GitHub environment | `pypi` |
 
-If production PyPI still has no project for this name, configure a pending publisher for a new project. If the project exists by the time production publishing is implemented, add the trusted publisher to the existing project instead.
-
-Do not create or document a PyPI API token for this workflow. Production upload should use Trusted Publishing / OIDC only.
+If production PyPI still has no project for this name, configure a pending
+publisher for a new project. If the project exists by the time production
+publishing is implemented, add the trusted publisher to the existing project
+instead.
+
+Do not create or document a PyPI API token for this workflow. Production upload
+should use Trusted Publishing / OIDC only.
 
 PyPI-side setup should use these paths:
 
-- for a new production project, create a pending publisher on production PyPI for project `sbom-diff-and-risk` with the owner, repository, workflow, and environment values above
-- for an existing production project, open that project on production PyPI and add a trusted publisher with the same owner, repository, workflow, and environment values
-- leave the environment field as `pypi`; if the PyPI publisher omits the environment, it will not match the future publish job identity
+- for a new production project, create a pending publisher on production PyPI
+  for project `sbom-diff-and-risk` with the owner, repository, workflow, and
+  environment values above
+- for an existing production project, open that project on production PyPI and
+  add a trusted publisher with the same owner, repository, workflow, and
+  environment values
+- leave the environment field as `pypi`; if the PyPI publisher omits the
+  environment, it will not match the future publish job identity
 - do not add a PyPI API token, PyPI password, or GitHub publishing secret as a fallback
 
 ## Prerequisites before enabling production publishing
 
 Before adding `.github/workflows/sbom-diff-and-risk-pypi.yml`, maintainers should complete all of these checks:
 
 - confirm the intended production package name still resolves as expected on production PyPI
-- choose the first production version, likely `0.5.0` or a later release tag
+- choose the first production version as an explicitly approved current or
+  future release tag
 - update `pyproject.toml` to that version
 - create or verify the matching GitHub tag and release assets
 - create the GitHub environment named `pypi`
@@ -90,7 +115,7 @@ The future production workflow should:
 - require an explicit boolean input such as `publish_to_pypi`
 - require a confirmation string such as `publish sbom-diff-and-risk to production PyPI`
 - require an expected version input and assert that it matches `pyproject.toml`
-- require the run ref to be a version tag such as `refs/tags/v0.5.0`
+- require the run ref to be a version tag such as `refs/tags/v<version>`
 - build the wheel and source distribution once
 - run `python -m twine check dist/*`
 - upload the checked distributions as a workflow artifact
@@ -99,11 +124,14 @@ The future production workflow should:
 - grant `id-token: write` only to the publish job
 - avoid production upload on ordinary push or pull request events
 
-The publish step should use `pypa/gh-action-pypi-publish@release/v1` without a `repository-url` override so it targets production PyPI.
+The publish step should use `pypa/gh-action-pypi-publish@release/v1` without a
+`repository-url` override so it targets production PyPI.
 
 ## Provenance boundaries
 
-Production PyPI Trusted Publishing provenance, GitHub workflow artifact attestations, and GitHub Release asset verification answer related but different questions.
+Production PyPI Trusted Publishing provenance, GitHub workflow artifact
+attestations, and GitHub Release asset verification answer related but
+different questions.
 
 PyPI Trusted Publishing provenance answers:
 
diff --git a/tools/sbom-diff-and-risk/docs/report-schema.md b/tools/sbom-diff-and-risk/docs/report-schema.md
@@ -71,6 +71,9 @@ Explanation fields appear only on policy finding objects. Risk findings in
 policy-decision metadata unless a policy evaluation maps them into policy
 findings.
 
+For reviewer-facing examples and interpretation guidance, see
+[policy-decision-explainability.md](policy-decision-explainability.md).
+
 ## Summary contract
 
 `summary` is the stable, compact entry point for automation that needs counts
diff --git a/tools/sbom-diff-and-risk/docs/reviewer-brief.md b/tools/sbom-diff-and-risk/docs/reviewer-brief.md
@@ -4,7 +4,7 @@
 
 `sbom-diff-and-risk` is a local CLI for comparing two SBOMs or dependency manifests and producing deterministic review artifacts: JSON, Markdown, and SARIF. It is built for conservative supply-chain review, not for vulnerability scanning or package reputation scoring.
 
-Current released version: `v0.6.0`.
+Current released version: `v0.7.0`.
 
 ## Why this project matters
 
@@ -28,6 +28,7 @@ Dependency review often needs evidence that is stable enough for code review, CI
 | What does the tool do? | `README.md`, examples, tests, and generated sample reports. |
 | How can a reviewer reproduce the core evidence? | [reviewer-evidence-pack.md](reviewer-evidence-pack.md) for demo, release, TestPyPI, and SARIF verification paths. |
 | What is the stable JSON shape? | [report-schema.md](report-schema.md) documents the machine-readable report structure and `summary` contract. |
+| How are policy findings explained? | [policy-decision-explainability.md](policy-decision-explainability.md) documents the policy decision metadata in JSON reports. |
 | Are default runs offline? | CLI docs, tests for no-enrichment behavior, and explicit enrichment flags. |
 | Can code scanning consume the output? | `docs/github-code-scanning.md` and `examples/sample-sarif.sarif`. |
 | Can the tool's own artifacts be verified? | `docs/self-provenance.md` for workflow artifact attestations. |
diff --git a/tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md b/tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md
diff --git a/tools/sbom-diff-and-risk/examples/github-actions-consumer.yml b/tools/sbom-diff-and-risk/examples/github-actions-consumer.yml
