feat: replace rank_packages() with cumulative-coverage impact scoring [CM-1228] by mbani01 · Pull Request #4204 · linuxfoundation/crowd.dev

mbani01 · 2026-06-12T12:06:17Z

This pull request updates the package criticality ranking system to use a new cumulative coverage-based approach and simplifies both the database schema and the worker script. The changes remove the previous weighted ranking and arbitrary top-N caps in favor of a more robust and interpretable method, and update the code to match the new logic and schema.

Database schema and ranking logic changes:

The package_criticality_spotlight table now uses a package_id foreign key instead of text-based identifiers, improving join performance and data integrity.
The rank_packages() function is rewritten to use cumulative coverage scoring: critical packages are those that together account for a specified percentage of each signal (e.g., downloads, dependents), and impact is calculated as the average of (1 − cumulative coverage) across signals. The function signature and logic are updated to support this.

Worker script updates:

The worker script (run-impact.ts) now accepts --cutoff and --ecosystems arguments instead of weights and top-N caps, aligning with the new ranking method. [1] [2]
The script's output and query parameters are updated to reflect the new function signature and return values.

Note

Medium Risk
Changes how is_critical, impact, and ecosystem ranks are computed for all ranked packages, and the spotlight migration requires existing rows to be valid package_ids before the NOT NULL FK applies.

Overview
Replaces package criticality ranking with cumulative-coverage scoring and aligns spotlight storage with integer package joins.

rank_packages() is dropped and recreated with (coverage_cutoff, ecosystems) instead of weighted signals and per-ecosystem top-N JSON. It ranks packages per ecosystem using downloads, direct dependents, and transitive dependents: is_critical is true when a package sits in the smallest prefix that reaches the cutoff on any non-zero signal (plus manual spotlight rows); impact is the mean of 1 − cumulative_share across applicable signals; rank_in_ecosystem orders by impact. It returns a single processed_rows count and bulk-updates packages.impact, is_critical, rank_in_ecosystem, and sync timestamps.

package_criticality_spotlight drops name/namespace and the old composite index, adds package_id (FK to packages) with a unique index, so the ranking pass joins spotlight by id.

run-impact.ts drops weight/top-N CLI flags in favor of --cutoff and --ecosystems, and logs processed_rows from the new function signature.

^{Reviewed by Cursor Bugbot for commit da9415e. Bugbot is set up for automated code reviews on this repo. Configure here.}

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

CLAassistant · 2026-06-12T12:06:24Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

cursor · 2026-06-12T12:07:59Z

+ALTER TABLE package_criticality_spotlight
+    ADD COLUMN package_id bigint NOT NULL REFERENCES packages(id),
+    DROP COLUMN name,
+    DROP COLUMN namespace;


Spotlight migration missing backfill

High Severity

The migration adds package_id as NOT NULL and drops namespace and name in one step, with no UPDATE to populate package_id from existing (ecosystem, namespace, name) rows first. On any database that already has package_criticality_spotlight rows, the migration fails and spotlight overrides cannot be migrated.

^{Reviewed by Cursor Bugbot for commit fdd2765. Configure here.}

cursor · 2026-06-12T12:08:00Z

+           last_rank_pass_at  = NOW(),
+           last_synced_at     = NOW()
+      FROM final
+     WHERE p.id = final.id;


Spotlight overrides scope reduced

Medium Severity

Spotlight forcing of is_critical now happens only inside the final join for packages that were ranked in the same pass. The previous rank_packages() ran a separate spotlight UPDATE for all matching packages, so partial --ecosystems runs (or packages skipped when signal totals are zero) can leave spotlight entries without is_critical = TRUE.

^{Reviewed by Cursor Bugbot for commit fdd2765. Configure here.}

Copilot

Pull request overview

Updates the packages criticality ranking pipeline to switch from weighted percentile ranking + top-N caps to a cumulative-coverage based scoring model, and adjusts both storage (spotlight overrides) and the packages_worker CLI accordingly.

Changes:

Replaces rank_packages() SQL logic with a cumulative-coverage approach and changes its signature to (coverage_cutoff, ecosystems), returning processed_rows.
Refactors package_criticality_spotlight to use a package_id FK instead of (ecosystem, namespace, name) matching.
Updates run-impact.ts CLI to accept --cutoff and --ecosystems and call the new SQL signature.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File	Description
`services/apps/packages_worker/src/criticality/run-impact.ts`	Updates the worker CLI parsing/logging and SQL invocation to match the new `rank_packages()` signature.
`backend/src/osspckgs/migrations/V1781262276__rank_packages_cumulative_coverage.sql`	Alters spotlight schema and replaces `rank_packages()` with cumulative-coverage scoring + bulk update of `packages` criticality fields.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+function parseListArg(flag: string): string[] | null {
  const idx = process.argv.indexOf(flag)
-  return idx !== -1 ? process.argv[idx + 1] : fallback
+  if (idx === -1) return null
+  return process.argv[idx + 1].split(',').map((s) => s.trim())
 }


…available signals Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 4 total unresolved issues (including 3 from previous reviews).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 8348b06. Configure here.}

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

+ALTER TABLE package_criticality_spotlight
+    ADD COLUMN IF NOT EXISTS package_id bigint NOT NULL REFERENCES packages(id),
+    DROP COLUMN IF EXISTS name,
+    DROP COLUMN IF EXISTS namespace;
+
+CREATE UNIQUE INDEX IF NOT EXISTS package_criticality_spotlight_package_id_idx
+    ON package_criticality_spotlight (package_id);


+function parseListArg(flag: string): string[] | null {
  const idx = process.argv.indexOf(flag)
-  return idx !== -1 ? process.argv[idx + 1] : fallback
+  if (idx === -1) return null
+  return process.argv[idx + 1].split(',').map((s) => s.trim())
 }


+const cutoff = parseArg('--cutoff', 0.9)
+const ecosystems = parseListArg('--ecosystems')



+DECLARE
+    processed_count      int;
+    effective_ecosystems text[];
+BEGIN


feat: replace rank_packages() with cumulative-coverage impact scoring

fdd2765

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

mbani01 self-assigned this Jun 12, 2026

Copilot AI review requested due to automatic review settings June 12, 2026 12:06

Copilot started reviewing on behalf of mbani01 June 12, 2026 12:06 View session

cursor Bot reviewed Jun 12, 2026

View reviewed changes

Copilot AI reviewed Jun 12, 2026

View reviewed changes

fix: compute impact as plain avg of (1 - cumulative_coverage) across …

b1a3674

…available signals Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

cursor Bot reviewed Jun 12, 2026

View reviewed changes

Comment thread backend/src/osspckgs/migrations/V1781262276__rank_packages_cumulative_coverage.sql Outdated

fix: make rank_packages migration idempotent with IF EXISTS guards

ddecc76

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

Copilot AI review requested due to automatic review settings June 12, 2026 15:43

Copilot started reviewing on behalf of mbani01 June 12, 2026 15:44 View session

mbani01 and others added 2 commits June 12, 2026 16:45

chore: format

3572d64

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

Merge branch 'main' into feat/impact_score_v2

8348b06

cursor Bot reviewed Jun 12, 2026

View reviewed changes

Comment thread backend/src/osspckgs/migrations/V1781262276__rank_packages_cumulative_coverage.sql

Copilot AI reviewed Jun 12, 2026

View reviewed changes

mbani01 requested a review from joanagmaia June 12, 2026 16:02

mbani01 changed the title ~~feat: replace rank_packages() with cumulative-coverage impact scoring~~ feat: replace rank_packages() with cumulative-coverage impact scoring [CM-1228] Jun 12, 2026

joanagmaia approved these changes Jun 15, 2026

View reviewed changes

Merge branch 'main' into feat/impact_score_v2

da9415e

Copilot AI review requested due to automatic review settings June 15, 2026 08:37

mbani01 merged commit 0a58453 into main Jun 15, 2026
11 checks passed

mbani01 deleted the feat/impact_score_v2 branch June 15, 2026 08:37

Copilot started reviewing on behalf of mbani01 June 15, 2026 08:37 View session

Copilot AI reviewed Jun 15, 2026

View reviewed changes

		const cutoff = parseArg('--cutoff', 0.9)
		const ecosystems = parseListArg('--ecosystems')

Uh oh!

Conversation

mbani01 commented Jun 12, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CLAassistant commented Jun 12, 2026

Uh oh!

cursor Bot Jun 12, 2026

Choose a reason for hiding this comment

Spotlight migration missing backfill

Uh oh!

Uh oh!

cursor Bot Jun 12, 2026

Choose a reason for hiding this comment

Spotlight overrides scope reduced

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mbani01 commented Jun 12, 2026 •

edited by cursor Bot

Loading