Skip to content

Commit df89604

Browse files
committed
refactor: fold osv-sync migrations into V1779710880 initial schema
Per Joana's preference (and consistent with how PRs #4148 and #4150 absorbed previous schema deltas), edit the consolidated initial migration in place rather than layering separate ones while packages-db is still pre-production. Folded into V1779710880: - packages.has_critical_vulnerability — boolean NOT NULL DEFAULT FALSE + partial index on TRUE. Replaces the deferred "column commented out" stub. Inline comment cites the option-b + MAL- semantics decision in ADR-0001 §`has_critical_vulnerability` semantics. - advisories.cvss_source — text, nullable, four documented values ('osv_cvss_v3', 'osv_cvss_v4', 'osv_qualitative_fallback', 'osv_malicious_package'). Inline comment cites ADR-0001 §CVSS scoring strategy and lists the values for downstream consumers. - advisory_affected_ranges unique index — widened from (advisory_package_id, COALESCE(introduced_version, '')) to the full tuple including COALESCE(fixed_version, '') and COALESCE(last_affected, ''). Inline comment cites ADR-0001 §`advisory_affected_ranges` uniqueness scope. Deleted (now folded): - V1779871303__add_cvss_source_to_advisories.sql - V1779871327__add_has_critical_vulnerability_to_packages.sql - V1779897650__widen_advisory_affected_ranges_unique_index.sql - V1779951727__align_advisory_ecosystem_lowercase.sql The lowercase ecosystem migration goes away entirely: the table is empty in prod, parseOsvRecord lowercases at the OSV boundary, and the application is the source of truth for the casing convention. No backfill needed. Local cost: one `flyway repair` (or volume drop + reapply) for any teammate with the branch checked out, since the V1779710880 checksum changes. Same pattern the team used twice this week. Verified locally: dropped the packages-db volume, ran Flyway from scratch, all expected columns and indexes present. The 70 unit tests still pass. Signed-off-by: Joan Reyero <joan@reyero.io>
1 parent de1fce6 commit df89604

5 files changed

Lines changed: 33 additions & 83 deletions

backend/src/osspckgs/migrations/V1779710880__initial_schema.sql

Lines changed: 33 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -58,10 +58,12 @@ CREATE TABLE packages (
5858
latest_release_at timestamptz,
5959
dependent_packages_count int,
6060
dependent_repos_count int,
61-
-- has_critical_vulnerability bool NOT NULL DEFAULT FALSE,
62-
-- Deferred: semantics undecided between (a) any advisory with no fixed_version vs
63-
-- (b) latest_version falls inside an affected semver range. Lateral join against
64-
-- advisory_packages used in queries until this is resolved.
61+
-- has_critical_vulnerability: TRUE iff latest_version is inside an active
62+
-- affected range of a critical advisory (CVSS >= 7.0) OR a MAL-* malicious-
63+
-- package advisory matches the package. Maintained by the deriveCriticalFlag
64+
-- activity in packages_worker/src/osv/. See ADR-0001 §`has_critical_vulnerability`
65+
-- semantics for the option-b + MAL- override rationale.
66+
has_critical_vulnerability bool NOT NULL DEFAULT FALSE,
6567
criticality_score numeric(10, 4),
6668
-- is_critical and last_rank_pass_at are not in the original pckgs.md spec; added so
6769
-- the packages table can answer "is this package critical?" without joining packages_universe,
@@ -82,8 +84,12 @@ CREATE INDEX ON packages (ecosystem, name);
8284

8385
CREATE INDEX ON packages USING gin (keywords);
8486

85-
-- INDEX on has_critical_vulnerability removed — column is commented out above.
86-
-- Uncomment both when semantics are decided.
87+
-- Partial index on has_critical_vulnerability TRUE rows only — that's the bucket
88+
-- the security overlay query needs ("list all packages with a known critical
89+
-- vuln"). The FALSE rows dominate the table and don't need an index.
90+
CREATE INDEX ON packages (has_critical_vulnerability)
91+
WHERE
92+
has_critical_vulnerability;
8793

8894
CREATE INDEX ON packages (criticality_score DESC)
8995
WHERE
@@ -569,6 +575,15 @@ CREATE TABLE advisories (
569575
aliases text[], -- CVE-XXXX, GHSA-...
570576
severity text, -- 'LOW' | 'MEDIUM' | 'HIGH' | 'CRITICAL'
571577
cvss numeric(3, 1),
578+
-- Provenance of the cvss value above. Lets downstream consumers distinguish
579+
-- a real vendor-supplied vector from a synthesized qualitative fallback.
580+
-- See ADR-0001 §CVSS scoring strategy. Allowed values:
581+
-- 'osv_cvss_v3' numeric score from a CVSS_V3 vector
582+
-- 'osv_cvss_v4' reserved; v4 numeric scoring deferred
583+
-- 'osv_qualitative_fallback' synthesized from database_specific.severity
584+
-- 'osv_malicious_package' MAL-* id with no CVSS vector
585+
-- Extensible to 'ghsa' | 'nvd' as additional sources come online.
586+
cvss_source text,
572587
-- >= 7.0 intentional: treat HIGH + CRITICAL both as actionable
573588
is_critical bool GENERATED ALWAYS AS (cvss >= 7.0) STORED,
574589
summary text,
@@ -613,7 +628,18 @@ CREATE TABLE advisory_affected_ranges (
613628
unaffected_raw text -- raw UnaffectedVersions string from deps.dev BQ
614629
);
615630

616-
CREATE UNIQUE INDEX ON advisory_affected_ranges (advisory_package_id, COALESCE(introduced_version, ''));
631+
-- Full-tuple uniqueness so two ranges sharing introduced_version but differing
632+
-- in fixed_version or last_affected (cross-distro patches, partial fixes in a
633+
-- single advisory) both survive insertion. The narrower (advisory_package_id,
634+
-- introduced_version) form silently collapsed those cases to one row, dropping
635+
-- the wider range and under-reporting vulnerable windows in the derive step.
636+
-- See ADR-0001 §`advisory_affected_ranges` uniqueness scope.
637+
CREATE UNIQUE INDEX ON advisory_affected_ranges (
638+
advisory_package_id,
639+
COALESCE(introduced_version, ''),
640+
COALESCE(fixed_version, ''),
641+
COALESCE(last_affected, '')
642+
);
617643

618644
CREATE INDEX ON advisory_affected_ranges (advisory_package_id);
619645

backend/src/osspckgs/migrations/V1779871303__add_cvss_source_to_advisories.sql

Lines changed: 0 additions & 10 deletions
This file was deleted.

backend/src/osspckgs/migrations/V1779871327__add_has_critical_vulnerability_to_packages.sql

Lines changed: 0 additions & 13 deletions
This file was deleted.

backend/src/osspckgs/migrations/V1779897650__widen_advisory_affected_ranges_unique_index.sql

Lines changed: 0 additions & 37 deletions
This file was deleted.

backend/src/osspckgs/migrations/V1779951727__align_advisory_ecosystem_lowercase.sql

Lines changed: 0 additions & 16 deletions
This file was deleted.

0 commit comments

Comments
 (0)