Skip to content

Commit 4c75c2d

Browse files
committed
fix: filter soft-deleted advisory ranges in Tinybird join (CM-1258)
Signed-off-by: Uroš Marolt <uros@marolt.me>
1 parent 2c031f9 commit 4c75c2d

2 files changed

Lines changed: 4 additions & 1 deletion

File tree

services/libs/tinybird/datasources/advisoryAffectedRanges.datasource

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ DESCRIPTION >
1010
- `rangeRaw` is the raw AffectedVersions string from the deps.dev BigQuery source (empty string if OSV-sourced).
1111
- `unaffectedRaw` is the raw UnaffectedVersions string from deps.dev BigQuery (empty string if OSV-sourced).
1212
- `createdAt` and `updatedAt` are row-level audit timestamps for Tinybird watermark-based sync.
13+
- `deletedAt` is set when Postgres soft-deletes the row (stale OSV tuple or superseded deps.dev raw row); NULL means live. `FINAL` alone doesn't collapse this — a soft-deleted row and its live replacement are distinct PKs, so consumers must also filter `deletedAt IS NULL` (see ADR-0001 §`advisory_affected_ranges` delete/dedup strategy, CM-1258).
1314

1415
SCHEMA >
1516
`id` UInt64 `json:$.record.id`,
@@ -20,7 +21,8 @@ SCHEMA >
2021
`rangeRaw` String `json:$.record.range_raw` DEFAULT '',
2122
`unaffectedRaw` String `json:$.record.unaffected_raw` DEFAULT '',
2223
`createdAt` DateTime64(3) `json:$.record.created_at`,
23-
`updatedAt` DateTime64(3) `json:$.record.updated_at`
24+
`updatedAt` DateTime64(3) `json:$.record.updated_at`,
25+
`deletedAt` Nullable(DateTime64(3)) `json:$.record.deleted_at`
2426

2527
ENGINE ReplacingMergeTree
2628
ENGINE_PARTITION_KEY toYear(createdAt)

services/libs/tinybird/pipes/ossPackages_enriched.pipe

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,7 @@ SQL >
9393
INNER JOIN advisoryPackages AS ap FINAL ON ap.packageId = p.id
9494
INNER JOIN advisories AS a FINAL ON a.id = ap.advisoryId
9595
INNER JOIN advisoryAffectedRanges AS aar FINAL ON aar.advisoryPackageId = ap.id
96+
AND aar.deletedAt IS NULL
9697
LEFT JOIN
9798
(SELECT packageId, number, publishedAt AS introducedAt FROM versions FINAL) AS vi
9899
ON vi.packageId = p.id

0 commit comments

Comments
 (0)