feat: Health Score v2 (Akrites) scoring pipeline, lifecycle, and impact score (CM-IN-1196)#4394
Open
gaspergrom wants to merge 8 commits into
Open
feat: Health Score v2 (Akrites) scoring pipeline, lifecycle, and impact score (CM-IN-1196)#4394gaspergrom wants to merge 8 commits into
gaspergrom wants to merge 8 commits into
Conversation
…(IN-1196) Signed-off-by: Gašper Grom <gasper.grom@gmail.com>
…althScore (IN-1196) Signed-off-by: Gašper Grom <gasper.grom@gmail.com>
- Split health_score_v2 into 5 category pipes plus a lightweight combiner, fixing a perf regression from re-scanning base tables per category - Fix graceful degradation: availability flags checked IS NOT NULL on non-Nullable columns, which ClickHouse fills with 0/'' not NULL on LEFT JOIN miss, so degradation never triggered - Fix Impact Score defaulting to 100 instead of NULL for repos with no packages (least(NULL, 100) = 100 in ClickHouse) - Fix Maintainer Responsiveness scoring per product review: GitHub/GitLab score 0 (not blocked) when no PR/issue data; Gerrit scores 0 (not blocked) when no changeset data, issues excluded since Gerrit has no issue tracker; excluded=true repos stay blocked, never scored 0 - project_insights_copy now reads split datasources via cheap LEFT JOIN Signed-off-by: Gašper Grom <gasper.grom@gmail.com>
maintainers_roles_copy_ds.endDate stays at the 1970-01-01 sentinel indefinitely unless someone explicitly closes the role, so archived/ dead repos retained a maxed-out busFactorScore=18 from maintainers with no commits or activity in years. Confirmed on Xamarin.Forms, HPCToolkit, Ursa, and SpaceVim, all archived, all showing healthScoreV2=100 after Layer 2 rescaling. Bus factor now also requires the maintainer to have activityRelations activity on that repo within the same trailing 12-month window already used for org diversity. Verified against live Tinybird data: Xamarin.Forms 7->0, SpaceVim 8->0, kubernetes/kubernetes 160->123 (unaffected), torvalds/linux 0->0 (unaffected). Signed-off-by: Gašper Grom <gasper.grom@gmail.com>
…(IN-1196) Per Joana's review: the lifecycle decision tree's final ELSE always resolved to 'active' whenever a repo failed every other branch, including repos with zero commits, zero issues, and zero PRs in every window checked. Confirmed on blocknetdx/blocknet: no commit-timeline data and exactly 0 activity everywhere, yet labeled 'active' purely because no other branch matched. health_score_v2_lifecycle.pipe now adds an explicit 'unavailable' branch before the active fallback, triggered only when commits are 0 in both 6-month windows AND issues+PRs are 0 in the 18-month window -- i.e. every signal this pipe looks at is exactly zero, not just one missing field. Repos with real activity that simply don't clear the abandoned/declining/stable thresholds (GravityView, Hitomi-Downloader, bminor/binutils-gdb -- all have substantial issue/PR activity despite missing repos.lastCommitAt or near-zero recent commits) correctly still resolve to 'active', unchanged. project_insights_copy.pipe's project-level best-state-wins rollup used a hardcoded 5-value priority array that didn't include 'unavailable' -- ClickHouse's indexOf() returns 0 for values not in the array, which would have sorted 'unavailable' as better than 'active' (index 1), letting one data-starved repo incorrectly override a genuinely active project. Added 'unavailable' explicitly as the lowest-priority state. Verified against live Tinybird data: blocknetdx/blocknet unavailable (was active); GravityView/Hitomi-Downloader/bminor-binutils-gdb still active (unchanged, real signal present); kubernetes/kubernetes unaffected (6331 commits last 6mo). Signed-off-by: Gašper Grom <gasper.grom@gmail.com>
Per Joana's follow-up: prefer NULL at the data layer for insufficient signal, consistent with how healthScoreV2 already represents it, and let the UI derive the Unavailable chip from NULL rather than carrying a literal string through the pipeline. health_score_v2_lifecycle.pipe: the zero-signal branch (zero commits in both 6-month windows AND zero issues/PRs in the 18-month window) now returns NULL instead of 'unavailable'. Archived repos are explicitly excluded from this check so they still correctly resolve to 'archived' even when they also have zero activity. Re-verified: blocknetdx/blocknet -> NULL (was 'unavailable', originally 'active'); xamarin/Xamarin.Forms -> 'archived' (unaffected); GravityView, Hitomi-Downloader, bminor/binutils-gdb -> 'active' (unaffected, real signal present). project_insights_copy.pipe: the project-level best-state-wins rollup no longer needs 'unavailable' in its priority array (back to the original 5 values), since groupArray() already drops NULL entries for free when at least one repo in the project has a real state. But an all-NULL or fully-excluded project makes groupArray() return an EMPTY array, and arrayElement([], 1) on an empty array silently returns an empty string, not NULL. Added an explicit empty-array guard so that case resolves to a real NULL too. While tracing this, found the actual root cause of 5 pre-existing blank-lifecycleLabel projects (Charliecloud, GNU Binutils and GDB, GravityView, GridGain Community Edition, Hitomi-Downloader): each project's sole repo has repositories.excluded = true, so the pipe's own WHERE rep.excluded = false filter drops every repo, groupArray() sees zero rows, and the old unguarded arrayElement returned ''. This empty-array guard fixes the class of bug generally; the excluded=true repo assignment for those 5 projects is a separate, pre-existing data situation, unrelated to IN-1196, not addressed here. Signed-off-by: Gašper Grom <gasper.grom@gmail.com>
Two chain breaks found during pre-deploy validation of 7a33bf7 that would have made the NULL-lifecycle fix fail outright or silently revert: health_score_v2_lifecycle_ds.datasource declared lifecycleLabelV2 as a plain String, not Nullable(String). Inserting NULL into a non-nullable ClickHouse column throws CANNOT_INSERT_NULL_IN_ORDINARY_COLUMN, so the lifecycle pipe's nightly copy job would hard-fail on its very first run after deploy, for any repo hitting the new NULL branch (confirmed: blocknetdx/blocknet is one right now). Widened to Nullable(String). health_score_v2.pipe wrapped the pass-through in coalesce(l.lifecycleLabelV2, 'active'), which would have silently converted the upstream NULL back into a fabricated 'active' one hop downstream, erasing the fix's effect before it ever reached health_score_v2_repo_copy_ds (whose own schema already correctly declares lifecycleLabelV2 as Nullable(String) — no change needed there). Confirmed live that this LEFT JOIN has zero actual row-misses across all repos, so the coalesce was never serving a real missing Signed-off-by: Gašper Grom <gasper.grom@gmail.com>
PR SummaryHigh Risk Overview Correctness fixes tighten scoring: maintainer bus factor now requires recent Separately, collection contributor dependency adds a daily Reviewed by Cursor Bugbot for commit a3e0359. Bugbot is set up for automated code reviews on this repo. Configure here. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
PR titles must follow Conventional Commits. Love from, Your reviewers ❤️.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| SELECT channel AS repoUrl, count(DISTINCT organizationId) AS orgCount | ||
| FROM activityRelations | ||
| WHERE timestamp > now() - INTERVAL 12 MONTH AND organizationId != '' | ||
| GROUP BY channel |
There was a problem hiding this comment.
Bus factor counts non-contribution activity
Medium Severity
The bus-factor fix requires recent activityRelations rows on the repo, but the subquery does not restrict activity type. Stars, forks, and other non-contribution events can satisfy the join, so maintainers with no real maintenance work in the window may still inflate bus factor—similar to the stale role-record issue this change was meant to fix.
Triggered by learned rule: Tinybird pipes counting contributors must filter by activityTypes
Reviewed by Cursor Bugbot for commit a20d1bc. Configure here.
| pm.activeContributorsPrevious365Days AS activeContributorsPrevious365Days, | ||
| pm.activeOrganizationsPrevious365Days AS activeOrganizationsPrevious365Days | ||
| pm.activeOrganizationsPrevious365Days AS activeOrganizationsPrevious365Days, | ||
| toUInt8(round(hv2.healthScoreV2)) AS healthScoreV2, |
There was a problem hiding this comment.
NULL health score becomes zero
High Severity
Project and repo rows set healthScoreV2 with toUInt8(round(hv2.healthScoreV2)) without a NULL guard, unlike upstream pipes that use toNullable. Missing or unavailable scores can become 0, which is a valid “critical” score and contradicts healthLabel when that field is NULL. When every included repo has NULL healthScoreV2, avg can yield NaN; NaN is not treated as NULL in the label multiIf, so the project can show healthLabel critical while the score is wrong.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit a20d1bc. Configure here.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds the Akrites Health Score v2 pipeline and exposes repository/project health, lifecycle, and impact metrics.
Changes:
- Materializes five independent scoring categories and combines available health signals.
- Adds lifecycle and impact scoring with project-level rollups.
- Exposes the new metrics through insights endpoints and datasources.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 16 comments.
Show a summary per file
| File | Description |
|---|---|
pipes/project_repo_insights.pipe |
Exposes Akrites fields for projects and repositories. |
pipes/project_insights.pipe |
Exposes Akrites project fields. |
pipes/project_insights_copy.pipe |
Rolls up and labels repository scores. |
pipes/health_score_v2.pipe |
Combines category scores with graceful degradation. |
pipes/health_score_v2_security.pipe |
Computes security and supply-chain health. |
pipes/health_score_v2_maintainer.pipe |
Computes maintainer health. |
pipes/health_score_v2_lifecycle.pipe |
Classifies repository lifecycle. |
pipes/health_score_v2_impact.pipe |
Computes package-mediated impact. |
pipes/health_score_v2_development.pipe |
Computes development activity health. |
datasources/project_insights_copy_ds.datasource |
Adds Akrites insight columns. |
datasources/health_score_v2_security_ds.datasource |
Stores security scores. |
datasources/health_score_v2_repo_copy_ds.datasource |
Stores combined repository results. |
datasources/health_score_v2_maintainer_ds.datasource |
Stores maintainer scores. |
datasources/health_score_v2_lifecycle_ds.datasource |
Stores lifecycle classifications. |
datasources/health_score_v2_impact_ds.datasource |
Stores raw impact scores. |
datasources/health_score_v2_development_ds.datasource |
Stores development scores. |
Comments suppressed due to low confidence (3)
services/libs/tinybird/pipes/health_score_v2_lifecycle.pipe:85
- All three values only use issues opened in the last 18 months, but the aggregation currently reads every partition in
issues_analyzed. Add the 18-month predicate toWHEREso ClickHouse can prune old yearly partitions.
FROM issues_analyzed
GROUP BY channel
services/libs/tinybird/pipes/health_score_v2_lifecycle.pipe:90
- The metric only counts PRs from the last 18 months, yet this scans all yearly partitions in
pull_requests_analyzed. Push the same time condition intoWHEREto prune historical partitions.
FROM pull_requests_analyzed
GROUP BY channel
services/libs/tinybird/datasources/health_score_v2_repo_copy_ds.datasource:24
- This statement contradicts the implemented Layer 2 graceful degradation in
health_score_v2.pipe: unavailable categories are excluded and remaining category scores are rescaled, while the result is NULL only when all categories are unavailable. The earlier “sum” description is also incomplete for that case.
- No graceful-degradation/signal-coverage redistribution yet — a repo with zero signal in a category
scores 0 there rather than having points redistributed from available categories.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| + if(d.developmentActivityScoreV2 IS NOT NULL, 25, 0)) AS coveredCategoryWeight, | ||
| l.lifecycleLabelV2 AS lifecycleLabelV2, | ||
| i.impactScoreRaw AS impactScoreRaw | ||
| FROM (SELECT DISTINCT url AS repoUrl FROM repositories WHERE deletedAt IS NULL) AS base |
| (rd.url != '') AS securityPracticesAvailable, | ||
| multiIf(coalesce(dh.vulnerableDeps,0) = 0, 5, dh.vulnerableDeps <= 2, 3, dh.vulnerableDeps <= 5, 1, 0) AS dependencyHealthScore, | ||
| (pkgs.repoUrl != '') AS dependencyHealthAvailable | ||
| FROM (SELECT DISTINCT url FROM repositories WHERE deletedAt IS NULL) AS allRepos |
| + multiIf(prs.medianMergeS < 604800, 2, prs.medianMergeS < 2592000, 1, 0)) AS prMergeScore | ||
| FROM ( | ||
| SELECT base.url AS repoUrl, rc.lastCommitAt AS lastCommitAt | ||
| FROM (SELECT DISTINCT url FROM repositories WHERE deletedAt IS NULL) AS base |
| SELECT | ||
| base.url AS repoUrl, | ||
| max(toFloat64OrNull(pk.impact)) * 100 AS impactScoreRaw | ||
| FROM (SELECT DISTINCT url FROM repositories WHERE deletedAt IS NULL) AS base |
| SELECT DISTINCT url AS repoUrl, | ||
| (domain(url) = 'review.opendev.org' OR domain(url) LIKE 'gerrit.%') AS isGerrit, | ||
| excluded AS isExcluded | ||
| FROM repositories WHERE deletedAt IS NULL |
| countIf(timestamp > now() - INTERVAL 6 MONTH) AS commitsLast6m, | ||
| countIf(timestamp <= now() - INTERVAL 6 MONTH AND timestamp > now() - INTERVAL 12 MONTH) AS commitsPrior6m | ||
| FROM activityRelations_deduplicated_cleaned_bucket_union | ||
| WHERE type = 'authored-commit' |
Comment on lines
+35
to
+38
| - `healthScoreV2` column is the Akrites-methodology composite health score (0-100), averaged from `ossPackages_enriched_ds.healthScore` across the project's linked packages. Distinct from the legacy `healthScore` column (community/contributor-based); null when the project has no linked package data. | ||
| - `healthLabel` column buckets `healthScoreV2`: 'excellent' (85+), 'healthy' (70-84), 'fair' (50-69), 'concerning' (30-49), 'critical' (<30). Lowercase, matches `ossPackages_enriched_ds.healthLabel` convention. | ||
| - `lifecycleLabel` column is the project's aggregated maintenance state across its packages, using best-state-wins precedence (active > stable > declining > abandoned > archived) over `ossPackages_enriched_ds.lifecycleLabel`. | ||
| - `impactScore` column is the average Osprey criticality `impact` (0-100) across the project's packages. Null when the project has no linked package data. |
| - `pageSize`: Optional integer for result limit, defaults to 10 | ||
| - `page`: Optional integer for pagination offset calculation, defaults to 0 | ||
| - Response: Project records with all insights metrics including achievements as array of (leaderboardType, rank, totalCount) tuples | ||
| - `healthScoreV2`, `healthLabel`, `lifecycleLabel`, `impactScore`, `impactLabel` are the Akrites-methodology fields (see `project_insights_copy_ds` for band definitions); may be null when the project has no linked package data. `healthScoreV2` is distinct from the legacy `healthScore` field. |
| - `pageSize`: Optional integer for result limit, defaults to 10 | ||
| - `page`: Optional integer for pagination offset calculation, defaults to 0 | ||
| - Response: Project and repository records with insights metrics | ||
| - `healthScoreV2`, `healthLabel`, `lifecycleLabel`, `impactScore`, `impactLabel` are the Akrites-methodology fields (see `project_insights_copy_ds` for band definitions); may be null when the record has no linked package data. `healthScoreV2` is distinct from the legacy `healthScore` field. |
Comment on lines
+11
to
+13
| - `securitySupplyChainScoreV2` (0-35) — open vulnerabilities, OpenSSF Scorecard, security practices, | ||
| dependency health (checked against the repo's own published packages' dependencies, not an average | ||
| across unrelated packages), supply chain integrity (hardcoded 0 — provenance/2FA data not yet piped). |
…uests CNCF (149,677 contributors) triggered a Tinybird production rate-limit alert (concurrency:1) on collection_contributor_dependency due to timeouts under load. The live computation must fully materialize a GROUP BY af.memberId over the entire collection's activity set before it can ORDER BY/LIMIT, which doesn't scale for large collections. Adds a daily copy pipe that precomputes the default (unfiltered) bus-factor result per collection. The serving pipe now reads the precomputed table when no optional filter is present (the frontend's first-paint shape) and falls through to the live path when a filter is set. Verified against production: CNCF requests dropped from 6-14s / ~150M rows scanned to ~20-130ms / 8,192 rows scanned. Signed-off-by: Gašper Grom <gasper.grom@gmail.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 3 potential issues.
There are 5 total unresolved issues (including 2 from previous reviews).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit a3e0359. Configure here.
| 'archived', | ||
| (r.lastCommitAt < now() - INTERVAL 18 MONTH) | ||
| AND coalesce(w.issuesInWindow18m, 0) + coalesce(p.prsInWindow18m, 0) = 0, | ||
| 'abandoned', |
There was a problem hiding this comment.
NULL lifecycle shadows abandoned
High Severity
The new no-signal NULL branch requires zero commits in the 12-month activityRelations windows and zero issues/PRs, but does not consider lastCommitAt. Repos with an old lastCommitAt and no issues/PRs — the normal abandoned case — also have zero ingested commits in those windows, so they hit NULL before the abandoned branch can run. In consistent data, abandoned becomes effectively unreachable.
Reviewed by Cursor Bugbot for commit a3e0359. Configure here.
| SELECT channel AS repoUrl, count(DISTINCT organizationId) AS orgCount | ||
| FROM activityRelations | ||
| WHERE timestamp > now() - INTERVAL 12 MONTH AND organizationId != '' | ||
| GROUP BY channel |
There was a problem hiding this comment.
Raw activityRelations in maintainer score
Medium Severity
Bus-factor and org-diversity joins read raw activityRelations, while the sibling Health Score v2 pipes and the datasource docs require activityRelations_deduplicated_cleaned_bucket_union for analytics. Uncleaned rows (duplicates, rows meant to be filtered out) can inflate recent-activity maintainer and org counts and skew Maintainer Health.
Reviewed by Cursor Bugbot for commit a3e0359. Configure here.
| INNER JOIN members_sorted m ON m.id = r.id | ||
| LEFT JOIN | ||
| (SELECT memberId, groupUniqArray(role) AS roles FROM maintainers_roles_copy_ds GROUP BY memberId) mr | ||
| ON mr.memberId = r.id |
There was a problem hiding this comment.
Roles not scoped to collection
Medium Severity
The precomputed copy aggregates maintainers_roles_copy_ds roles by memberId only, with no collection/insightsProjectId filter. The live path uses member_roles, which restricts roles to projects in the collection. Precomputed responses can show maintainer roles from outside the collection, so default (unfiltered) widget loads disagree with the filtered live path.
Reviewed by Cursor Bugbot for commit a3e0359. Configure here.
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 19 changed files in this pull request and generated 5 comments.
Comments suppressed due to low confidence (8)
services/libs/tinybird/pipes/health_score_v2_maintainer.pipe:81
repositoriesis a versionedReplacingMergeTree, but this query filtersdeletedAtand reads mutableexcludedbefore collapsing versions. A repository whose exclusion flag changed can therefore produce two rows with the same URL and duplicate the category output. ReadFINAL, as established inproject_insights_copy.pipe:131andrepositories_populated_copy.pipe:7.
FROM repositories WHERE deletedAt IS NULL
services/libs/tinybird/pipes/health_score_v2.pipe:52
- Because
repositoriesis a versionedReplacingMergeTree, filtering beforeFINALleaves old non-deleted versions visible and can retain soft-deleted repositories in the final score datasource. Use the current version before applyingdeletedAt, consistent withrepositories_populated_copy.pipe:7.
FROM (SELECT DISTINCT url AS repoUrl FROM repositories WHERE deletedAt IS NULL) AS base
services/libs/tinybird/pipes/health_score_v2_lifecycle.pipe:60
- This reads mutable
archivedvalues from every physical version of therepositoriesReplacingMergeTree. If a repository was archived or unarchived,DISTINCT url, archivedemits both states and materializes conflicting lifecycle rows for one URL. Collapse withFINALbefore filtering/selecting mutable fields.
FROM (SELECT DISTINCT url, archived FROM repositories WHERE deletedAt IS NULL) AS base
services/libs/tinybird/pipes/project_insights_copy.pipe:176
- The displayed project score is rounded to an integer, but the label thresholds use the unrounded average. For example, 84.6 is exposed as
healthScoreV2 = 85while receivinghealthLabel = 'healthy', contradicting the documented 85+ “excellent” band. Apply the thresholds to the same rounded value.
toUInt8(round(hv2.healthScoreV2)) AS healthScoreV2,
multiIf(
hv2.healthScoreV2 IS NULL,
NULL,
hv2.healthScoreV2 >= 85,
services/libs/tinybird/datasources/project_insights_copy_ds.datasource:39
- These datasource descriptions document the previous package-derived calculation, but the new pipe computes health from per-repository category scores, lifecycle across repository labels, and impact as a maximum rather than an average. This is the contract consumers will consult, so update it to match the implemented aggregation and null conditions.
- `healthScoreV2` column is the Akrites-methodology composite health score (0-100), averaged from `ossPackages_enriched_ds.healthScore` across the project's linked packages. Distinct from the legacy `healthScore` column (community/contributor-based); null when the project has no linked package data.
- `healthLabel` column buckets `healthScoreV2`: 'excellent' (85+), 'healthy' (70-84), 'fair' (50-69), 'concerning' (30-49), 'critical' (<30). Lowercase, matches `ossPackages_enriched_ds.healthLabel` convention.
- `lifecycleLabel` column is the project's aggregated maintenance state across its packages, using best-state-wins precedence (active > stable > declining > abandoned > archived) over `ossPackages_enriched_ds.lifecycleLabel`.
- `impactScore` column is the average Osprey criticality `impact` (0-100) across the project's packages. Null when the project has no linked package data.
- `impactLabel` column buckets `impactScore`: 'foundational' (85-100), 'major' (60-84), 'moderate' (30-59), 'minor' (0-29). Null when `impactScore` is null.
services/libs/tinybird/pipes/project_insights.pipe:15
- This public endpoint description ties all five fields to linked package data, but health and lifecycle are repository-derived and become null based on signal coverage; only impact is package-mediated. Update the nullability note so API consumers do not infer the wrong prerequisite.
- `healthScoreV2`, `healthLabel`, `lifecycleLabel`, `impactScore`, `impactLabel` are the Akrites-methodology fields (see `project_insights_copy_ds` for band definitions); may be null when the project has no linked package data. `healthScoreV2` is distinct from the legacy `healthScore` field.
services/libs/tinybird/pipes/project_repo_insights.pipe:14
- This endpoint description says the Akrites fields may be null due to missing package data, but repository health and lifecycle do not depend on packages. Only impact has that prerequisite; health/lifecycle nullability reflects unavailable repository signals.
- `healthScoreV2`, `healthLabel`, `lifecycleLabel`, `impactScore`, `impactLabel` are the Akrites-methodology fields (see `project_insights_copy_ds` for band definitions); may be null when the record has no linked package data. `healthScoreV2` is distinct from the legacy `healthScore` field.
services/libs/tinybird/datasources/health_score_v2_repo_copy_ds.datasource:24
- This states that graceful degradation is not implemented, directly contradicting
health_score_v2.pipe:33-49, which rescales over available category weights. The earlier “sum” description is consequently also inaccurate whenever a category is unavailable. Document the actual rescaling behavior in one place.
- No graceful-degradation/signal-coverage redistribution yet — a repo with zero signal in a category
scores 0 there rather than having points redistributed from available categories.
Comment on lines
+84
to
+93
| SELECT mr.repoUrl AS repoUrl, count(DISTINCT mr.memberId) AS busFactorCount | ||
| FROM maintainers_roles_copy_ds mr | ||
| INNER JOIN ( | ||
| SELECT DISTINCT memberId, channel AS repoUrl | ||
| FROM activityRelations | ||
| WHERE timestamp > now() - INTERVAL 12 MONTH | ||
| ) AS recentActivity | ||
| ON recentActivity.memberId = mr.memberId AND recentActivity.repoUrl = mr.repoUrl | ||
| WHERE mr.endDate = '1970-01-01 00:00:00' OR mr.endDate > now() - INTERVAL 12 MONTH | ||
| GROUP BY mr.repoUrl |
Comment on lines
+145
to
+147
| LEFT JOIN | ||
| (SELECT memberId, groupUniqArray(role) AS roles FROM maintainers_roles_copy_ds GROUP BY memberId) mr | ||
| ON mr.memberId = r.id |
| collectionSlug | ||
| = {{ String(collectionSlug, description="Filter by collection slug", required=True) }} | ||
| AND isBusFactorCutoff = 1 | ||
| ORDER BY rank ASC |
Comment on lines
+149
to
+151
| TYPE COPY | ||
| TARGET_DATASOURCE collection_contributor_dependency_copy_ds | ||
| COPY_MODE replace |
|
|
||
| SQL > | ||
| % | ||
| {% if not defined(repos) and not defined(startDate) and not defined(endDate) and not defined(platform) and not defined(activity_type) and not Boolean(includeCollaborations, false) and Int32(limit, 100) == 100 %} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the Health Score v2 (Akrites methodology) scoring pipeline: per-repo maintainer health, security/supply-chain, and development-activity category scores, plus a lifecycle classifier and an impact score, rolled up to project level. Also fixes several correctness bugs found during data verification against production before rollout.
healthScoreV2applies spec-defined graceful degradation: if 1-2 of 3 categories are unavailable, the score rescales across only the available categories rather than defaulting to a fabricated 0; NULL only when all three are unavailable.endDate(a manually-curated field that stays at a "still active" sentinel indefinitely unless someone explicitly closes the role). This let archived/dead repos keep a maxed-out bus-factor score from maintainers with no real activity in years, which combined with the graceful-degradation rescale to producehealthScoreV2 = 100on dead repos. Bus factor now also requires the maintainer to have actualactivityRelationsactivity on that repo within the same trailing 12-month window. Verified against production: Xamarin.Forms and SpaceVim (archived) drop from a maxed bus-factor score to 0; kubernetes/kubernetes stays healthy and unaffected.ELSEbranch always resolved to'active'whenever a repo failed every other branch, including repos with zero commits, zero issues, and zero PRs in every window checked (confirmed live on a fully dormant, non-archived repo with no commit-timeline data and no signal at all). The chain now returnsNULLfor that case instead of a fabricated'active', propagated correctly through:health_score_v2_lifecycle_ds— widenedlifecycleLabelV2fromStringtoNullable(String)(was going to hard-fail the copy job on any NULL insert).health_score_v2.pipe— removed acoalesce(lifecycleLabelV2, 'active')that would have silently erased the NULL back into'active'one hop downstream.project_insights_copy.pipe— the project-level "best-state-wins" rollup already drops NULL entries for free viagroupArray, but an all-NULL or fully-excluded project produces an empty array, andarrayElement([], 1)on an empty array silently returns''rather than NULL. Added an explicit guard. While tracing this, found the actual root cause of 5 pre-existing blank-lifecycleLabelprojects (each project's sole repo hasrepositories.excluded = true) — a separate, pre-existing data situation, not fixed here, but no longer surfaces as a silent empty string.All fixes verified against live production Tinybird data before merge, including full-chain propagation checks and regression checks against known-healthy projects (kubernetes, projects with real but thin activity signal).
Test plan
tb checkpasses on all changed pipes/datasourceshealthScoreV2=100to realistic scores (22-62); active repos (kubernetes, GravityView, Hitomi-Downloader, binutils-gdb) unaffected'archived'even with zero activity; repos with any real signal still resolve to a real state, not NULLString→Nullable(String)) is a non-breaking, metadata-only ClickHouse ALTERDeploy notes
Deploy order matters — datasources before pipes, then pipes in dependency order:
health_score_v2_lifecycle_ds.datasource,health_score_v2_repo_copy_ds.datasource(schema changes)health_score_v2_lifecycle.pipe,health_score_v2_maintainer.pipe(independent, both feedhealth_score_v2)health_score_v2.pipe(reads both category datasources above)project_insights_copy.pipe(readshealth_score_v2_repo_copy_ds)After pushing pipes, the copy jobs need a forced run (
tbc <pipe> --yes) rather than waiting for the nightly cron, in the same order, so the NULL/bus-factor fixes propagate through the full chain before this is user-visible.