Skip to content

Commit e8e9fc8

Browse files
committed
feat(scan): grade end-of-life findings by how close they are
--block-eol was binary: anything already past its end-of-life date failed the build, and anything not yet past it was invisible. A runtime that died two years ago and one that dies next quarter are not the same problem, and a gate that cannot tell them apart gets switched off. Each EOL date is now mapped to a synthetic severity by its horizon (retired, within 30 days, this quarter, next quarter) using the organisation's own mapping from its quality-gate policy. Those four eol*Severity columns have existed, with sensible defaults, since the quality-gate table was created, and nothing has ever read them -- because until organisations were seeded a policy row, cli.quality-gate-get answered {"config": null} for every org alive and the CLI never got as far as looking. A bucket set to "skip" produces nothing at all. Blocking behaviour is deliberately unchanged. Only findings at or above the block floor fail the build, and the floor defaults to critical, which is the retired bucket's default severity -- so exactly what failed a build yesterday fails it today, and everything closer to the horizon is reported rather than enforced. Grading arrived with a CLI release, not with any org's decision to adopt it, and a release that turned "your runtime dies next quarter" into a red build would break CI for every customer who has ever passed --block-eol. --block-eol-severity opts into more.
1 parent b9f392f commit e8e9fc8

6 files changed

Lines changed: 558 additions & 23 deletions

File tree

.repo/policy-fetch.md

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Policy Fetch
2+
3+
Last Updated: 2026-07-14
4+
5+
The CLI fetches three kinds of organisation policy from vdb-api. This document covers what each fetch does with the answer, and (the part worth reading) what `{"config": null}` used to mean and no longer does.
6+
7+
Server side: [vdb-api/.repo/auth-and-tenancy.md](https://github.com/vulnetix/vdb-api/blob/main/.repo/auth-and-tenancy.md). Identity model: [vulnetix-authentic-aws/.repo/identity-model.md](https://github.com/vulnetix/vulnetix-authentic-aws/blob/main/.repo/identity-model.md).
8+
9+
## The three fetches
10+
11+
| Endpoint | Client method (`pkg/vdb/api_cli.go`) | Consumers |
12+
| --- | --- | --- |
13+
| `POST /v2/cli.quality-gate-get` | `CliQualityGateGet` | `applyOrgQualityGate` at scan time (`cmd/quality_gate_enforce.go`); `config get quality-gate` (`renderQualityGateGet`) |
14+
| `POST /v2/cli.package-firewall-get` | `CliPackageFirewallGet` | `config get package-firewall` (`renderPackageFirewallGet`) |
15+
| `POST /v2/cli.ai-firewall-baseline` | `CliAiFirewallBaseline` | `ai-firewall baseline`; composed into `ai-firewall apply` when `spec.baseline.enabled` |
16+
17+
All three are read-only, all three carry an empty payload. The org is resolved from the authenticated request, and none of them creates anything server-side.
18+
19+
The uuid in the API key identifies a **principal**. The policy that comes back belongs to that principal's **tenant**, resolved server-side by `TenantOrgID`. A Teams member and their owner run the same `vulnetix scan` and are gated by one policy, which is the entire point of the split.
20+
21+
## Quality gate at scan time
22+
23+
```mermaid
24+
sequenceDiagram
25+
participant Scan as vulnetix scan
26+
participant AQG as applyOrgQualityGate
27+
participant API as vdb-api
28+
participant SA as SaaS Postgres
29+
30+
Scan->>AQG: qualityGateOverridePointers{9 scan-time locals}
31+
AQG->>AQG: auth.LoadCredentials()
32+
alt no creds, or community tier
33+
AQG-->>Scan: no-op, scan flags and builtin defaults stand
34+
end
35+
AQG->>API: POST /v2/cli.quality-gate-get (ApiKey = principal)
36+
API->>API: TenantOrgID(principal) → tenant
37+
API->>SA: SELECT ... FROM "CliQualityGateConfig" WHERE "orgUuid" = tenant
38+
alt lookup fails
39+
API-->>AQG: error
40+
AQG-->>Scan: no-op, scan flags stand (verbose notes it)
41+
end
42+
SA-->>API: row (or none)
43+
API-->>AQG: {"config": {...}} or {"config": null}
44+
45+
alt config == null
46+
AQG-->>Scan: "no policy configured", scan flags stand
47+
else config present
48+
loop each of the 9 enforcement fields
49+
AQG->>AQG: field null? → leave the caller's local alone
50+
AQG->>AQG: field set? → OVERWRITE the local; ORG POLICY ALWAYS WINS
51+
end
52+
AQG-->>Scan: locals now carry the org's enforcement
53+
end
54+
```
55+
56+
Two semantics are decided and should not be relitigated without a reason.
57+
58+
**Org policy always wins.** A value the org has set overrides even an explicitly-passed CLI flag. `noteOverride` prints `--severity high superseded by org policy: critical` under `--verbose`. A security policy a developer can turn off with a flag is not a policy.
59+
60+
**NULL means "not set", not "false".** `qgConfigBool` / `qgConfigInt` / `qgConfigString` each return a second boolean that is false when the field is absent or JSON `null`, and the caller then leaves its local untouched. This is what makes the seeded defaults safe: a freshly-provisioned org has all nine enforcement columns NULL, so its CI behaves exactly as it did before it had a policy row at all.
61+
62+
## What `{"config": null}` used to mean
63+
64+
It meant **every organisation**, and it meant that for the entire life of the product.
65+
66+
Nothing ever seeded a `CliQualityGateConfig` row. Not `register`, not the OIDC callback, not the Stripe webhook. The three doors each did their own thing and none of them wrote a line of policy. So `POST /v2/cli.quality-gate-get` returned `{"config": null}` for every org that had ever existed, and `applyOrgQualityGate` took its "no policy configured" early return every single time. The four EOL-to-severity columns (`eolNextQuarterSeverity`, `eolThisQuarterSeverity`, `eolWithin30DaysSeverity`, `eolRetiredSeverity`), all with perfectly good defaults, were unreachable dead schema. The feature was wired end to end and could not fire.
67+
68+
Since `ProvisionOrg` ([vdb-site/.repo/org-provisioning.md](https://github.com/vulnetix/vdb-site/blob/main/.repo/org-provisioning.md)), a new org is seeded at creation with the EOL buckets populated and the nine enforcement columns left NULL. So today:
69+
70+
- **`{"config": null}`** means an org that predates the seeder and has not been backfilled. It is now the exception, not the rule.
71+
- **`{"config": {...}}` with every enforcement field null** means a seeded org that has not opted into enforcement. This is the normal state of a new org, and it is why an upgrade does not suddenly start failing builds that used to pass. Enforcement is a decision a human makes.
72+
- **`{"config": {...}}` with fields set** means an owner configured it, and it wins.
73+
74+
The CLI needs no change for any of this: `renderQualityGateGet` already prints a "no policy configured" line for a nil config and "not set" for each null field, and `applyOrgQualityGate` already treats the two cases differently. What changed is which branch actually gets taken.
75+
76+
## Package firewall
77+
78+
`CliPackageFirewallGet` returns `{"config": {...}|null, "mirrors": [...]}`. For a nil config, `renderPackageFirewallGet` reports that no policy is configured and that the proxy defaults apply, which is now literally true: the proxy's in-memory defaults are the same open values the columns declare, and package-firewall no longer writes a row on a read ([package-firewall/.repo/org-policy-resolution.md](https://github.com/vulnetix/package-firewall/blob/main/.repo/org-policy-resolution.md)).
79+
80+
A seeded org has a permissive config row (every threshold `0`, every block `false`) plus the mirrors package-firewall backfills for each ecosystem. The `mirrors` array is populated from the tenant's config row, so every member of an org sees the same mirror set.
81+
82+
## AI firewall baseline
83+
84+
`CliAiFirewallBaseline` fetches the server's recommended guardrail set. It is composed into `ai-firewall apply` when `spec.baseline.enabled` is true, and it **soft-fails on 404** unless `--baseline-required`. An older server simply may not have the endpoint.
85+
86+
The same baseline is what vdb-site seeds onto a new org, and this is deliberate: vdb-site embeds the output of vdb-api's `cmd/baseline-export` rather than keeping a hand-copied second list, so "baseline 2026.07.1" cannot be one set of rules over the wire and a different set in the database. 23 guardrails, 21 enabled; `injection-delimiter-spoof` and `pii-phone` ship disabled because a guardrail that cries wolf teaches an organisation to switch the firewall off.
87+
88+
The practical consequence for `apply`: on a freshly provisioned org, the guardrails the server reports as the baseline are already present as rows, so a first `apply` with the baseline composed in is close to a no-op rather than a bulk create. Guardrails reconcile by **name**, which is what makes that idempotent.
89+
90+
## Related documents
91+
92+
- [vdb-api/.repo/auth-and-tenancy.md](https://github.com/vulnetix/vdb-api/blob/main/.repo/auth-and-tenancy.md). How the API key becomes a tenant, and the 401/403/409/503 rules the CLI's retry logic sees.
93+
- [vdb-site/.repo/org-provisioning.md](https://github.com/vulnetix/vdb-site/blob/main/.repo/org-provisioning.md). What gets seeded and why the defaults are asymmetric.
94+
- [vulnetix-authentic-aws/.repo/identity-model.md](https://github.com/vulnetix/vulnetix-authentic-aws/blob/main/.repo/identity-model.md), the hub.

cmd/quality_gate_enforce.go

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ import (
1515
"os"
1616

1717
"github.com/spf13/cobra"
18+
"github.com/vulnetix/cli/v3/internal/scan"
1819
"github.com/vulnetix/cli/v3/pkg/auth"
1920
"github.com/vulnetix/cli/v3/pkg/vdb"
2021
)
@@ -33,6 +34,12 @@ type qualityGateOverridePointers struct {
3334
exploits *string
3435
severity *string
3536
scaAutofixStrategy *string
37+
38+
// eolBuckets is the org's EOL-horizon-to-severity mapping. Unlike the nine
39+
// fields above it is NOT nullable enforcement — the columns are NOT NULL and
40+
// carry defaults — so it is not an override but a setting, and it is only read
41+
// when --block-eol is in effect.
42+
eolBuckets *scan.EOLSeverityBuckets
3643
}
3744

3845
// applyOrgQualityGate fetches the authenticated org's quality-gate policy and
@@ -108,6 +115,26 @@ func applyOrgQualityGate(cmd *cobra.Command, p qualityGateOverridePointers) {
108115
noteOverride(cmd, flag, callerVal, orgVal)
109116
}
110117

118+
// The EOL severity buckets. These four columns have existed, with sensible
119+
// defaults, since the quality-gate table was created — and nothing has ever
120+
// read them, because until organisations were seeded a config row, this handler
121+
// returned {"config": null} for every org on the platform and we never got this
122+
// far. They grade an EOL horizon instead of treating every EOL alike.
123+
if p.eolBuckets != nil {
124+
if v, ok := qgConfigString(config, "eolRetiredSeverity"); ok {
125+
p.eolBuckets.Retired = v
126+
}
127+
if v, ok := qgConfigString(config, "eolWithin30DaysSeverity"); ok {
128+
p.eolBuckets.Within30Days = v
129+
}
130+
if v, ok := qgConfigString(config, "eolThisQuarterSeverity"); ok {
131+
p.eolBuckets.ThisQuarter = v
132+
}
133+
if v, ok := qgConfigString(config, "eolNextQuarterSeverity"); ok {
134+
p.eolBuckets.NextQuarter = v
135+
}
136+
}
137+
111138
applyBool("block-eol", "blockEol", p.blockEol)
112139
applyBool("block-malware", "blockMalware", p.blockMalware)
113140
applyBool("block-unpinned", "blockUnpinned", p.blockUnpinned)

cmd/scan.go

Lines changed: 116 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,29 @@ type GateBreach struct {
6464
Message string // pre-formatted, ready to print
6565
}
6666

67+
// orgEOLBuckets maps an end-of-life horizon to a severity. It starts at the same
68+
// defaults the CliQualityGateConfig columns declare, and applyOrgQualityGate
69+
// overwrites it with the org's own mapping when the scan is authenticated.
70+
//
71+
// Package-level for the same reason `verbose` is: the gate that consumes it sits
72+
// behind a thirty-argument positional signature, and threading a thirty-first
73+
// through it would obscure the change rather than clarify it.
74+
var orgEOLBuckets = scan.DefaultEOLSeverityBuckets()
75+
76+
// eolBlockSeverity is the floor at which a graded EOL finding actually FAILS the
77+
// build, as opposed to merely being reported.
78+
//
79+
// It defaults to `critical`, which is the default severity of the `retired`
80+
// bucket — so by default exactly what failed a build before grading existed fails
81+
// it now: components already past their end-of-life date. Everything closer to the
82+
// horizon is reported and does not break anyone's pipeline.
83+
//
84+
// This default is deliberately conservative. Grading arrived with the CLI release,
85+
// not with an org's decision to adopt it, and a release that turns "your runtime
86+
// dies next quarter" into a red build would break CI for every customer who has
87+
// ever passed --block-eol.
88+
var eolBlockSeverity = "critical"
89+
6790
// MultiPolicyBreachError is returned when one or more quality gates are breached.
6891
type MultiPolicyBreachError struct {
6992
Breaches []GateBreach
@@ -401,6 +424,9 @@ func runScanWithFeatures(ctx context.Context, cmd *cobra.Command, noSAST, noSCA,
401424

402425
blockMalware, _ := cmd.Flags().GetBool("block-malware")
403426
blockEOL, _ := cmd.Flags().GetBool("block-eol")
427+
if v, _ := cmd.Flags().GetString("block-eol-severity"); v != "" {
428+
eolBlockSeverity = strings.ToLower(strings.TrimSpace(v))
429+
}
404430
blockUnpinned, _ := cmd.Flags().GetBool("block-unpinned")
405431
exploitThreshold, _ := cmd.Flags().GetString("exploits")
406432
resultsOnly, _ := cmd.Flags().GetBool("results-only")
@@ -432,6 +458,7 @@ func runScanWithFeatures(ctx context.Context, cmd *cobra.Command, noSAST, noSCA,
432458
// / community scans and orgs without a policy leave these values untouched.
433459
applyOrgQualityGate(cmd, qualityGateOverridePointers{
434460
blockEol: &blockEOL,
461+
eolBuckets: &orgEOLBuckets,
435462
blockMalware: &blockMalware,
436463
blockUnpinned: &blockUnpinned,
437464
cooldown: &cooldownDays,
@@ -1795,34 +1822,71 @@ func runLocalScan(
17951822
}
17961823

17971824
// Gate 4: EOL — best-effort runtime version pin detection + VDB EOL API.
1825+
//
1826+
// GRADED for reporting, but the BLOCKING behaviour is unchanged. Each
1827+
// end-of-life date is mapped to a synthetic severity by how close it is
1828+
// (retired / within 30 days / this quarter / next quarter) using the org's own
1829+
// mapping, so a runtime that died two years ago and one that dies next quarter
1830+
// stop looking like the same problem. A bucket the org set to "skip" produces
1831+
// nothing at all.
1832+
//
1833+
// Only components AT OR ABOVE the block floor actually fail the build, and the
1834+
// floor defaults to `critical`, which is the `retired` bucket's default
1835+
// severity. So out of the box exactly what failed a build yesterday fails it
1836+
// today, and everything else is reported rather than enforced. Grading a
1837+
// next-quarter EOL into a red build would turn an upgrade of the CLI into a
1838+
// fleet-wide CI outage, and a gate that does that gets switched off — which
1839+
// protects nobody.
1840+
//
1841+
// An org opts into stricter blocking with --block-eol-severity (high, medium,
1842+
// low). Doing it through the org's quality-gate policy needs a column that does
1843+
// not exist yet; that is a follow-up, not a reason to ship a breaking default.
1844+
//
1845+
// This is what the four eol*Severity columns were always for. Nothing read them
1846+
// until organisations started being seeded a policy row — before that,
1847+
// cli.quality-gate-get answered {"config": null} for every org alive and the
1848+
// CLI never got as far as looking.
17981849
if blockEOL {
17991850
eolClient := newSearchClient()
18001851
pins := scan.DetectRuntimeVersionPins(rootPath)
1801-
var eolViolations []string
1852+
now := time.Now()
1853+
1854+
type eolItem struct {
1855+
label string
1856+
severity string
1857+
horizon scan.EOLHorizon
1858+
}
1859+
var graded []eolItem
1860+
1861+
grade := func(eolFrom, label string) {
1862+
horizon := scan.EOLHorizonOf(eolFrom, now)
1863+
severity, ok := orgEOLBuckets.SeverityFor(horizon)
1864+
if !ok {
1865+
return
1866+
}
1867+
graded = append(graded, eolItem{label: label, severity: severity, horizon: horizon})
1868+
}
1869+
18021870
for _, pin := range pins {
18031871
resp, err := eolClient.EOLRelease(pin.Product, pin.Release)
18041872
if err != nil {
18051873
continue // silently skip: unknown product / network error
18061874
}
1807-
if resp.Release.IsEol {
1808-
eolViolations = append(eolViolations, fmt.Sprintf("%s %s (%s)", pin.Product, pin.RawVersion, pin.SourceFile))
1875+
eolFrom := ""
1876+
if resp.Release.EolFrom != nil {
1877+
eolFrom = *resp.Release.EolFrom
18091878
}
1810-
}
1811-
if len(eolViolations) > 0 {
1812-
breaches = append(breaches, GateBreach{
1813-
Gate: "eol",
1814-
Count: len(eolViolations),
1815-
Message: fmt.Sprintf("--block-eol: %s end-of-life: %s",
1816-
pluralise("runtime", len(eolViolations)),
1817-
strings.Join(eolViolations, ", ")),
1818-
})
1879+
// An EOL feed that says "this is EOL" but gives no date still means EOL.
1880+
if eolFrom == "" && resp.Release.IsEol {
1881+
eolFrom = now.Format("2006-01-02")
1882+
}
1883+
grade(eolFrom, fmt.Sprintf("%s %s (%s)", pin.Product, pin.RawVersion, pin.SourceFile))
18191884
}
18201885

18211886
// Package-level EOL — from /v2/cli.sca PackageInsights (the server maps
18221887
// each package to its EolProduct/EolRelease and matches the installed
18231888
// version's release line). Packages with no EOL row are skipped.
18241889
seenPkgEol := map[string]bool{}
1825-
var eolPkgList []string
18261890
for _, p := range allPackages {
18271891
if p.Version == "" {
18281892
continue
@@ -1833,24 +1897,52 @@ func runLocalScan(
18331897
}
18341898
seenPkgEol[purl] = true
18351899
ins, ok := insightByPurl[purl]
1836-
if !ok || !ins.IsEOL {
1900+
if !ok {
18371901
continue
18381902
}
1839-
label := fmt.Sprintf("%s@%s (%s)", p.Name, p.Version, p.Ecosystem)
1840-
if ins.EOLFrom != "" {
1841-
label = fmt.Sprintf("%s@%s (%s, EOL since %s)", p.Name, p.Version, p.Ecosystem, ins.EOLFrom)
1903+
eolFrom := ins.EOLFrom
1904+
if eolFrom == "" && ins.IsEOL {
1905+
eolFrom = now.Format("2006-01-02")
18421906
}
1843-
eolPkgList = append(eolPkgList, label)
1907+
grade(eolFrom, fmt.Sprintf("%s@%s (%s)", p.Name, p.Version, p.Ecosystem))
18441908
}
1845-
if len(eolPkgList) > 0 {
1909+
1910+
// One breach per severity, so the report reads as a priority list rather
1911+
// than one undifferentiated pile. Only severities at or above the floor
1912+
// breach; the rest are reported and do not fail the build.
1913+
var approaching []eolItem
1914+
for _, severity := range []string{"critical", "high", "medium", "low"} {
1915+
var labels []string
1916+
for _, item := range graded {
1917+
if item.severity != severity {
1918+
continue
1919+
}
1920+
if !scan.SeverityMeetsThreshold(severity, eolBlockSeverity) {
1921+
approaching = append(approaching, item)
1922+
1923+
continue
1924+
}
1925+
labels = append(labels, item.label)
1926+
}
1927+
if len(labels) == 0 {
1928+
continue
1929+
}
18461930
breaches = append(breaches, GateBreach{
18471931
Gate: "eol",
1848-
Count: len(eolPkgList),
1849-
Message: fmt.Sprintf("--block-eol: %s end-of-life: %s",
1850-
pluralise("package", len(eolPkgList)),
1851-
strings.Join(eolPkgList, ", ")),
1932+
Count: len(labels),
1933+
Message: fmt.Sprintf("--block-eol (%s): %s end-of-life: %s",
1934+
severity, pluralise("component", len(labels)), strings.Join(labels, ", ")),
18521935
})
18531936
}
1937+
1938+
// Approaching end-of-life. Not a breach — said plainly so it can be planned
1939+
// for, rather than discovered on the day it starts failing the build.
1940+
if len(approaching) > 0 && !resultsOnly {
1941+
fmt.Fprintf(os.Stderr, "\nApproaching end-of-life (not blocking; raise with --block-eol-severity):\n")
1942+
for _, item := range approaching {
1943+
fmt.Fprintf(os.Stderr, " %-14s %s (%s)\n", item.horizon, item.label, item.severity)
1944+
}
1945+
}
18541946
}
18551947

18561948
// Gate 5: unpinned direct dependencies
@@ -4175,6 +4267,7 @@ func addScanFlags(cmd *cobra.Command) {
41754267
cmd.Flags().Bool("block-malware", false, "Exit with code 1 when any dependency is a known malicious package.")
41764268
cmd.Flags().Bool("no-malscan", false, "Skip the in-process malscan-engine pass over local dependency install dirs.")
41774269
cmd.Flags().Bool("block-eol", false, "Exit with code 1 when a runtime or package dependency is end-of-life. Runtimes: Go, Node.js, Python, Ruby. Package-level checks activate when VDB has EOL data (404s are silently skipped).")
4270+
cmd.Flags().String("block-eol-severity", "critical", "With --block-eol, the graded severity at which an end-of-life component fails the build (critical, high, medium, low). Components graded below it are reported, not blocked. The default blocks only what is already past its end-of-life date.")
41784271
cmd.Flags().Bool("block-unpinned", false, "Exit with code 1 when any direct dependency uses a version range (^, ~, >=) instead of an exact pin.")
41794272
cmd.Flags().String("exploits", "", "Exit with code 1 when exploit maturity reaches the threshold: poc (any public exploit), active (CISA/EU KEV / actively exploited), weaponized (in-the-wild only).")
41804273
cmd.Flags().Bool("results-only", false, "Only output when findings exist; completely silent when the scan is clean.")

0 commit comments

Comments
 (0)