feat: health Score v2 (Akrites) scoring pipeline, lifecycle, and impact score (CM-IN-1196) (#4394)

Signed-off-by: Gašper Grom <gasper.grom@gmail.com>

Author: gaspergrom

services/libs/tinybird/datasources/collection_contributor_dependency_copy_ds.datasource

@@ -0,0 +1,53 @@
+DESCRIPTION >
+    - `collection_contributor_dependency_copy_ds` contains precomputed bus-factor (contributor
+    dependency) results for the DEFAULT, unfiltered view of every collection - i.e. the exact
+    output `collection_contributor_dependency.pipe` would compute today with no `repos`/`startDate`/
+    `endDate`/`platform`/`activity_type`/`includeCollaborations` override and `limit=100`.
+    - Created via `collection_contributor_dependency_copy.pipe` (`TYPE COPY`, daily schedule) to
+    avoid recomputing the same expensive per-member GROUP BY over the full activity set on every
+    request - see that pipe's DESCRIPTION for the full rationale (this materialization exists
+    because the live computation triggered a Tinybird production rate-limit/concurrency alert on
+    `lfx_insights` from timeouts under load).
+    - One row per (collectionSlug, memberId) that made it into that collection's top-100 ranked
+    contributors, already carrying the running-total percentage and the 51% cutoff membership flag
+    so the serving pipe only needs a cheap point lookup, not a recompute.
+    - `collectionSlug` is the collection this row belongs to.
+    - `id` is the member id (contributor).
+    - `displayName` is the member's display name.
+    - `githubHandleArray` is the member's GitHub usernames.
+    - `contributionCount` is the member's raw contribution count within the collection (all-time,
+    all-repos, code contributions only - the default filter set).
+    - `contributionPercentage` is `contributionCount` as a percentage of the collection's total
+    contribution count.
+    - `roles` is the member's maintainer roles within the collection, if any.
+    - `contributionPercentageRunningTotal` is the cumulative percentage when contributors are
+    ordered by `contributionPercentage DESC, id` - matches the live pipe's window function output.
+    - `rank` is the 1-indexed rank of this contributor within the collection (1 = highest
+    contributionCount), preserved so the serving pipe can `ORDER BY rank` cheaply instead of
+    re-sorting by percentage.
+    - `totalContributorCount` is the collection's total unique contributor count (same value
+    repeated on every row for the collection) - matches `active_contributors.contributorCount` in
+    the live pipe's response.
+    - `isBusFactorCutoff` marks the rows the live pipe's WHERE clause would return (running total
+    <= 51% OR running total minus this row's own percentage < 51%) - the serving pipe filters on
+    this instead of recomputing the cutoff logic.
+    - `updatedAt` is when this row's collection was last (re)computed - lets the serving pipe or an
+    operator sanity-check freshness independent of the COPY job's own execution log.
+
+SCHEMA >
+    `collectionSlug` String,
+    `id` String,
+    `displayName` String,
+    `githubHandleArray` Array(String),
+    `contributionCount` UInt64,
+    `contributionPercentage` Float64,
+    `roles` Array(String),
+    `contributionPercentageRunningTotal` Float64,
+    `rank` UInt32,
+    `totalContributorCount` UInt64,
+    `isBusFactorCutoff` UInt8,
+    `updatedAt` DateTime64(3)
+
+ENGINE ReplacingMergeTree
+ENGINE_SORTING_KEY collectionSlug, rank
+ENGINE_VER updatedAt

services/libs/tinybird/datasources/health_score_v2_development_ds.datasource

@@ -0,0 +1,13 @@
+DESCRIPTION >
+    - `health_score_v2_development_ds` holds the per-repo Development Activity category score (0-25)
+    for Health Score v2. Populated by `health_score_v2_development.pipe`.
+    - `repoUrl` is the repository URL — the join key back to `repositories`.
+    - `developmentActivityScoreV2` — NULL when covered sub-signal weight is <40% of the 25pt max
+    (spec Layer 1 graceful degradation), otherwise the rescaled 0-25 score.
+
+SCHEMA >
+    `repoUrl` String,
+    `developmentActivityScoreV2` Nullable(UInt8)
+
+ENGINE MergeTree
+ENGINE_SORTING_KEY repoUrl

services/libs/tinybird/datasources/health_score_v2_impact_ds.datasource

@@ -0,0 +1,13 @@
+DESCRIPTION >
+    - `health_score_v2_impact_ds` holds the per-repo raw Impact Score (0-100 float, pre-rounding)
+    for Health Score v2. Populated by `health_score_v2_impact.pipe`.
+    - `repoUrl` is the repository URL — the join key back to `repositories`.
+    - `impactScoreRaw` — MAX(packages.impact) * 100 over packages published by this repo. NULL when
+    the repo has no linked packages with a known impact value.
+
+SCHEMA >
+    `repoUrl` String,
+    `impactScoreRaw` Nullable(Float64)
+
+ENGINE MergeTree
+ENGINE_SORTING_KEY repoUrl

services/libs/tinybird/datasources/health_score_v2_lifecycle_ds.datasource

@@ -0,0 +1,14 @@
+DESCRIPTION >
+    - `health_score_v2_lifecycle_ds` holds the per-repo Lifecycle state for Health Score v2.
+    Populated by `health_score_v2_lifecycle.pipe`.
+    - `repoUrl` is the repository URL — the join key back to `repositories`.
+    - `lifecycleLabelV2` — one of active/stable/declining/abandoned/archived, or NULL when the
+    repo has no usable activity signal at all (2026-07-24, IN-1196 — see
+    health_score_v2_lifecycle.pipe for the exact condition).
+
+SCHEMA >
+    `repoUrl` String,
+    `lifecycleLabelV2` Nullable(String)
+
+ENGINE MergeTree
+ENGINE_SORTING_KEY repoUrl

services/libs/tinybird/datasources/health_score_v2_maintainer_ds.datasource

@@ -0,0 +1,13 @@
+DESCRIPTION >
+    - `health_score_v2_maintainer_ds` holds the per-repo Maintainer Health category score (0-40)
+    for Health Score v2. Populated by `health_score_v2_maintainer.pipe`.
+    - `repoUrl` is the repository URL — the join key back to `repositories`.
+    - `maintainerHealthScoreV2` — NULL when covered sub-signal weight is <40% of the 40pt max
+    (spec Layer 1 graceful degradation), otherwise the rescaled 0-40 score.
+
+SCHEMA >
+    `repoUrl` String,
+    `maintainerHealthScoreV2` Nullable(UInt8)
+
+ENGINE MergeTree
+ENGINE_SORTING_KEY repoUrl

services/libs/tinybird/datasources/health_score_v2_repo_copy_ds.datasource

@@ -0,0 +1,36 @@
+DESCRIPTION >
+    - `health_score_v2_repo_copy_ds` contains the per-repo Akrites Health Score v2 breakdown.
+    - Populated by `health_score_v2.pipe` copy pipe.
+    - Health Score v2 is computed entirely from repo-level/CDP data (maintainer activity, security posture,
+    development activity) — no package data, per the Health Score v2 methodology
+    (https://linuxfoundation-dxwx.dsp.so/4ezMyw7n-insights-projects-new-scores). Impact Score is the
+    one exception — package-mediated by design, computed here as MAX(packages.impact) over the repo's
+    own published packages (via packageRepos), not an average over unrelated packages.
+    - `repoUrl` is the repository URL — the join key back to `repositories`/`repos`.
+    - `maintainerHealthScoreV2` (0-40) — bus factor, org diversity, maintainer responsiveness.
+    - `securitySupplyChainScoreV2` (0-35) — open vulnerabilities, OpenSSF Scorecard, security practices,
+    dependency health (checked against the repo's own published packages' dependencies, not an average
+    across unrelated packages), supply chain integrity (hardcoded 0 — provenance/2FA data not yet piped).
+    - `developmentActivityScoreV2` (0-25) — release cadence, commit activity, issue resolution, PR merge health.
+    - `healthScoreV2` (0-100) is the sum of the three categories above, clamped to 100.
+    - `lifecycleLabelV2` — per-repo lifecycle state (active/stable/declining/abandoned/archived), computed
+    via the spec's decision tree (archived flag > abandoned > declining > stable > active, first match
+    wins), or NULL when the repo has zero commits and zero issues/PRs in every window checked (2026-07-24,
+    IN-1196 — no usable activity signal at all). Project-level rollup uses best-state-wins precedence in
+    project_insights_copy.pipe.
+    - `impactScore` (0-100) — MAX(packages.impact) * 100 over packages published by this repo. NULL when
+    the repo has no linked packages.
+    - No graceful-degradation/signal-coverage redistribution yet — a repo with zero signal in a category
+    scores 0 there rather than having points redistributed from available categories.
+
+SCHEMA >
+    `repoUrl` String,
+    `maintainerHealthScoreV2` Nullable(UInt8),
+    `securitySupplyChainScoreV2` Nullable(UInt8),
+    `developmentActivityScoreV2` Nullable(UInt8),
+    `healthScoreV2` Nullable(UInt8),
+    `lifecycleLabelV2` Nullable(String),
+    `impactScore` Nullable(UInt8)
+
+ENGINE MergeTree
+ENGINE_SORTING_KEY repoUrl

services/libs/tinybird/datasources/health_score_v2_security_ds.datasource

@@ -0,0 +1,13 @@
+DESCRIPTION >
+    - `health_score_v2_security_ds` holds the per-repo Security & Supply Chain category score (0-35)
+    for Health Score v2. Populated by `health_score_v2_security.pipe`.
+    - `repoUrl` is the repository URL — the join key back to `repositories`.
+    - `securitySupplyChainScoreV2` — NULL when covered sub-signal weight is <40% of the 35pt max
+    (spec Layer 1 graceful degradation), otherwise the rescaled 0-35 score.
+
+SCHEMA >
+    `repoUrl` String,
+    `securitySupplyChainScoreV2` Nullable(UInt8)
+
+ENGINE MergeTree
+ENGINE_SORTING_KEY repoUrl

services/libs/tinybird/datasources/project_insights_copy_ds.datasource

@@ -32,6 +32,11 @@ DESCRIPTION >
     - `activeContributorsPrevious365Days` column is the unique count of active contributors in the previous 365 days (365-730 days ago).
     - `activeOrganizationsPrevious365Days` column is the unique count of active organizations in the previous 365 days (365-730 days ago).
     - `status` column is the current status of the project (e.g., 'active', 'archived').
+    - `healthScoreV2` column is the Akrites-methodology composite health score (0-100), averaged from `ossPackages_enriched_ds.healthScore` across the project's linked packages. Distinct from the legacy `healthScore` column (community/contributor-based); null when the project has no linked package data.
+    - `healthLabel` column buckets `healthScoreV2`: 'excellent' (85+), 'healthy' (70-84), 'fair' (50-69), 'concerning' (30-49), 'critical' (<30). Lowercase, matches `ossPackages_enriched_ds.healthLabel` convention.
+    - `lifecycleLabel` column is the project's aggregated maintenance state across its packages, using best-state-wins precedence (active > stable > declining > abandoned > archived) over `ossPackages_enriched_ds.lifecycleLabel`.
+    - `impactScore` column is the average Osprey criticality `impact` (0-100) across the project's packages. Null when the project has no linked package data.
+    - `impactLabel` column buckets `impactScore`: 'foundational' (85-100), 'major' (60-84), 'moderate' (30-59), 'minor' (0-29). Null when `impactScore` is null.
 
 TAGS "Project insights", "Metrics"
 
@@ -65,7 +70,12 @@ SCHEMA >
     `starsPrevious365Days` UInt64,
     `forksPrevious365Days` UInt64,
     `activeContributorsPrevious365Days` UInt64,
-    `activeOrganizationsPrevious365Days` UInt64
+    `activeOrganizationsPrevious365Days` UInt64,
+    `healthScoreV2` Nullable(UInt8),
+    `healthLabel` Nullable(String),
+    `lifecycleLabel` Nullable(String),
+    `impactScore` Nullable(UInt8),
+    `impactLabel` Nullable(String)
 
 ENGINE MergeTree
 ENGINE_SORTING_KEY type, id

services/libs/tinybird/pipes/collection_contributor_dependency.pipe

@@ -8,25 +8,66 @@ DESCRIPTION >
     replacement for that table is `collection_contributors_leaderboard` (a differently-shaped
     pipe - see its own DESCRIPTION for why a naive collectionSlug branch on the single-project
     leaderboard pipe isn't used there either).
-    - Combines data from `collection_contributors_leaderboard` and `active_contributors` (both
-    already collectionSlug-aware) to provide dependency risk assessment at collection scale.
+    - **Two paths, branched on whether any optional filter is present:**
+    - **Precomputed path (no `repos`/`startDate`/`endDate`/`platform`/`activity_type` and
+    `includeCollaborations` absent or `false`)** - this is what the real frontend widget always
+    requests on first paint, before a user touches a filter control. Reads
+    `collection_contributor_dependency_copy_ds`, a daily `TYPE COPY` materialization (see
+    `collection_contributor_dependency_copy.pipe`), instead of recomputing live. This exists
+    because the live computation - via `collection_contributors_leaderboard`'s
+    `collection_member_aggregates` node - must fully materialize a `GROUP BY af.memberId` over the
+    entire filtered activity set before it can ORDER BY/LIMIT (LIMIT only reduces output size, not
+    scan/aggregation cost), which at collection scale (e.g. `cncf` has 149,677 distinct
+    contributors) triggered a Tinybird production rate-limit/concurrency alert on `lfx_insights`
+    from timeouts under load.
+    - **Live path (any of those filters present)** - unavoidable, since filters are genuine dynamic
+    user input and the precomputed table only covers the unfiltered default. Unchanged from the
+    prior implementation: combines `collection_contributors_leaderboard` and `active_contributors`
+    (both already collectionSlug-aware).
+    - Precomputed path only supports `limit=100` (the only value the frontend ever sends for this
+    widget, and the only value the copy pipe materializes) - if `limit` is explicitly set to
+    anything else, falls through to the live path even with no other filters present.
     - Parameters:
     - `collectionSlug`: Required collection slug (e.g., 'cncf') - inherited from `segments_filtered_by_collection`.
-    - `repos`: Optional array of repository URLs for filtering (inherited from `segments_filtered_by_collection`)
-    - `startDate`: Optional DateTime filter for activities after timestamp (e.g., '2024-01-01 00:00:00')
-    - `endDate`: Optional DateTime filter for activities before timestamp (e.g., '2024-12-31 23:59:59')
-    - `platform`: Optional string filter for source platform (e.g., 'github', 'discord', 'slack')
-    - `activity_type`: Optional string filter for single activity type (e.g., 'authored-commit')
-    - `includeCollaborations`: Optional boolean, defaults to false - same toggle as the single-project pipe.
+    - `repos`: Optional array of repository URLs for filtering (inherited from `segments_filtered_by_collection`). Presence routes to the live path.
+    - `startDate`: Optional DateTime filter for activities after timestamp (e.g., '2024-01-01 00:00:00'). Presence routes to the live path.
+    - `endDate`: Optional DateTime filter for activities before timestamp (e.g., '2024-12-31 23:59:59'). Presence routes to the live path.
+    - `platform`: Optional string filter for source platform (e.g., 'github', 'discord', 'slack'). Presence routes to the live path.
+    - `activity_type`: Optional string filter for single activity type (e.g., 'authored-commit'). Presence routes to the live path.
+    - `includeCollaborations`: Optional boolean, defaults to false - same toggle as the single-project pipe. `true` routes to the live path.
     - `limit`: Optional integer for the number of top contributors to rank before finding the 51%
     cutoff. Inherited by `collection_contributors_leaderboard` (this pipe has no default of its
     own), whose own default is 10 if omitted - callers doing bus-factor analysis should pass
     limit=100 explicitly (same convention as the single-project `contributor_dependency.pipe` /
-    `contributors_leaderboard.pipe` pair) to reliably reach the 51% cutoff.
+    `contributors_leaderboard.pipe` pair) to reliably reach the 51% cutoff. A value other than 100
+    routes to the live path.
     - Response: `id`, `displayName`, `githubHandleArray`, `contributionCount`, `contributionPercentage`, `roles`, `contributionPercentageRunningTotal`, `totalContributorCount`
 
 TAGS "Insights, Widget", "Collection", "Contributors"
 
+NODE collection_contributor_dependency_precomputed
+DESCRIPTION >
+    Precomputed path: point lookup into the daily materialization, filtered to the rows the live
+    pipe's WHERE clause would itself return (`isBusFactorCutoff = 1`), ordered by rank.
+
+SQL >
+    %
+    SELECT
+        id,
+        displayName,
+        githubHandleArray,
+        contributionCount,
+        contributionPercentage,
+        roles,
+        contributionPercentageRunningTotal,
+        totalContributorCount
+    FROM collection_contributor_dependency_copy_ds
+    WHERE
+        collectionSlug
+        = {{ String(collectionSlug, description="Filter by collection slug", required=True) }}
+        AND isBusFactorCutoff = 1
+    ORDER BY rank ASC
+
 NODE collection_contributions_percentage_running_total
 SQL >
     %
@@ -50,3 +91,19 @@ SQL >
         contributionPercentageRunningTotal <= 51
         OR (contributionPercentageRunningTotal - contributionPercentage < 51)
     ORDER BY contributionPercentageRunningTotal ASC
+
+NODE collection_contributor_dependency_result
+DESCRIPTION >
+    Routes between the precomputed and live paths. Precomputed path applies only when no optional
+    filter is present and `limit` (if given) is exactly 100 - the exact shape the frontend always
+    requests on first paint.
+
+SQL >
+    %
+    {% if not defined(repos) and not defined(startDate) and not defined(endDate) and not defined(
+        platform
+    ) and not defined(activity_type) and not Boolean(includeCollaborations, false) and Int32(
+        limit, 100
+    ) == 100 %} SELECT * FROM collection_contributor_dependency_precomputed
+    {% else %} SELECT * FROM collection_contributions_percentage_running_total
+    {% end %}