feat: scope widget pipes by collection for aggregate tabs (IN-1195)

services/libs/tinybird/pipes/active_contributors.pipe

@@ -4,7 +4,8 @@ DESCRIPTION >
     - **When `granularity` is NOT provided, returns total unique contributor count** for the specified filters.
     - **When `granularity` is provided, returns time-series data** showing unique contributor counts for each time period.
     - Parameters:
-    - `project`: Required string for project slug (e.g., 'k8s', 'tensorflow') - inherited from `segments_filtered`
+    - `project`: Required string for project slug (e.g., 'k8s', 'tensorflow') - inherited from `segments_filtered`. Required unless `collectionSlug` is given.
+    - `collectionSlug`: Collection slug (e.g., 'cncf') - inherited from `segments_filtered_by_collection`. Required unless `project` is given.
     - `repos`: Optional array of repository URLs for filtering (inherited from `segments_filtered`)
     - `startDate`: Optional DateTime filter for activities after timestamp (e.g., '2024-01-01 00:00:00')
     - `endDate`: Optional DateTime filter for activities before timestamp (e.g., '2024-12-31 23:59:59')
@@ -55,7 +56,10 @@ SQL >
     SELECT uniqIf(memberId, memberId != '') AS maintainerCount
     FROM maintainers_roles_copy_ds
     WHERE
-        insightsProjectId = (SELECT insightsProjectId FROM segments_filtered) AND role = 'maintainer'
+        {% if defined(collectionSlug) %}
+            insightsProjectId IN (SELECT insightsProjectId FROM segments_filtered_by_collection)
+        {% else %} insightsProjectId = (SELECT insightsProjectId FROM segments_filtered)
+        {% end %} AND role = 'maintainer'
         {% if defined(repos) %}
             AND repoUrl
             IN {{ Array(repos, 'String', description="Filter maintainer repo list", required=False) }}

services/libs/tinybird/pipes/activities_filtered.pipe

@@ -2,9 +2,12 @@ DESCRIPTION >
     - `activities_filtered.pipe` is the core filtering infrastructure pipe for activity data across the entire analytics platform.
     - This pipe serves as the foundation for most activity-related widgets, by providing a consistent, filtered view of contribution activities.
     - It filters activities from `activityRelations_deduplicated_cleaned_ds` datasource based on project segment, time ranges, repositories, platforms, and activity types.
-    - The pipe automatically scopes data to the current project using `segments_filtered` pipe for security and data isolation.
+    - The pipe scopes data either to a single project (via `segments_filtered`) or to every project in a collection
+    (via `segments_filtered_by_collection`), for security and data isolation. Exactly one of `project`/`collectionSlug`
+    must be provided by the caller.
     - Parameters:
-    - `project`: Inherited from `segments_filtered`, project slug (e.g., 'k8s', 'tensorflow')
+    - `project`: Project slug (e.g., 'k8s', 'tensorflow') - inherited from `segments_filtered`. Required unless `collectionSlug` is given.
+    - `collectionSlug`: Collection slug (e.g., 'cncf') - inherited from `segments_filtered_by_collection`. Required unless `project` is given.
     - `repos`: Optional array of repository URLs for filtering (e.g., ['https://github.com/kubernetes/kubernetes']). Inherited from `segments_filtered`.
     - `startDate`: Optional DateTime filter for activities after timestamp (e.g., '2024-01-01 00:00:00')
     - `endDate`: Optional DateTime filter for activities before timestamp (e.g., '2024-12-31 23:59:59')
@@ -21,9 +24,15 @@ NODE activities_filtered_bucket_routing
 SQL >
     %
     SELECT activityId as id, timestamp, type, platform, memberId, organizationId, segmentId
-    FROM activityRelations_bucket_routing a
+    FROM
+        {% if defined(collectionSlug) %} activityRelations_deduplicated_cleaned_bucket_union
+        {% else %} activityRelations_bucket_routing
+        {% end %} as a
     where
-        segmentId = (SELECT segmentId FROM segments_filtered)
+        {% if defined(collectionSlug) %}
+            segmentId IN (SELECT segmentId FROM segments_filtered_by_collection)
+        {% else %} segmentId = (SELECT segmentId FROM segments_filtered)
+        {% end %}
         {% if defined(startDate) %}
             AND a.timestamp
             > {{ DateTime(startDate, description="Filter activity timestamp after", required=False) }}

services/libs/tinybird/pipes/activities_filtered_retention.pipe

@@ -5,7 +5,8 @@ DESCRIPTION >
     - This time extension ensures retention rate calculations have the necessary baseline data from the preceding period for comparing distinct member activity for the first given period.
     - Primary use case: providing data foundation for member retention analysis and cohort studies.
     - Parameters:
-    - `project`: Inherited from `segments_filtered`, project slug (e.g., 'k8s', 'tensorflow')
+    - `project`: Inherited from `segments_filtered`, project slug (e.g., 'k8s', 'tensorflow'). Required unless `collectionSlug` is given.
+    - `collectionSlug`: Inherited from `segments_filtered_by_collection`, collection slug (e.g., 'cncf'). Required unless `project` is given.
     - `repos`: Inherited from `segments_filtered`, array of repository URLs for filtering
     - `startDate`: Optional DateTime filter, automatically extended backwards by one granularity period
     - `endDate`: Optional DateTime filter for activities before timestamp (e.g., '2024-12-31 23:59:59')
@@ -22,9 +23,15 @@ NODE activities_filtered_retention_bucket_routing
 SQL >
     %
     SELECT activityId as id, timestamp, type, platform, memberId, organizationId, segmentId
-    FROM activityRelations_bucket_routing a
+    FROM
+        {% if defined(collectionSlug) %} activityRelations_deduplicated_cleaned_bucket_union
+        {% else %} activityRelations_bucket_routing
+        {% end %} as a
     where
-        segmentId = (SELECT segmentId FROM segments_filtered)
+        {% if defined(collectionSlug) %}
+            segmentId IN (SELECT segmentId FROM segments_filtered_by_collection)
+        {% else %} segmentId = (SELECT segmentId FROM segments_filtered)
+        {% end %}
         {% if defined(startDate) %}
             AND a.timestamp
             > {% if defined(granularity) and granularity == "daily" %}

services/libs/tinybird/pipes/activityTypes_by_project.pipe

@@ -1,9 +1,15 @@
 DESCRIPTION >
-    - `activityTypes_by_project.pipe` returns activity types that actually exist for a given project.
+    - `activityTypes_by_project.pipe` returns activity types that actually exist for a given project, or
+    across every project in a collection.
     - Unlike `activityTypes_filtered` which returns all possible activity types, this pipe only returns types with actual activities in the project.
     - Useful for populating dropdowns, filters, or analytics that should only show activity types with data.
     - Parameters:
-    - `project`: **Required** string for project slug (e.g., 'k8s', 'tensorflow'). Passed to `segments_filtered`.
+    - `project`: String for project slug (e.g., 'k8s', 'tensorflow'). Passed to `segments_filtered`. Required unless `collectionSlug` is given.
+    - `collectionSlug`: Collection slug (e.g., 'cncf') - inherited from `segments_filtered_by_collection`. Required unless `project` is given.
+    Collection requests read from `activityRelations_deduplicated_cleaned_bucket_union` (all 10 buckets) instead
+    of `activityRelations_bucket_routing` (a single bucket, defaulting to bucket 0 when `bucketId` is not
+    injected), since a collection's member projects' segments can be routed to any of the 10 buckets - same
+    pattern as `activities_filtered.pipe`.
     - `repos`: Optional array of repository URLs for filtering (e.g., ['https://github.com/kubernetes/kubernetes']). Filters activities by repository.
     - `includeCodeContributions`: Optional boolean to include code contribution activities. Defaults to 1. Set to 0 to exclude. Passed to `activityTypes_filtered`.
     - `includeCollaborations`: Optional boolean to include or exclude collaboration activities. Defaults to 0. Passed to `activityTypes_filtered`.
@@ -15,10 +21,16 @@ NODE activityTypes_by_project_0
 SQL >
     %
     SELECT DISTINCT a.type as activityType, a.platform, at.label
-    FROM activityRelations_bucket_routing a
+    FROM
+        {% if defined(collectionSlug) %} activityRelations_deduplicated_cleaned_bucket_union
+        {% else %} activityRelations_bucket_routing
+        {% end %} a
     INNER JOIN activityTypes at ON a.type = at.activityType AND a.platform = at.platform
     WHERE
-        a.segmentId = (SELECT segmentId FROM segments_filtered)
+        {% if defined(collectionSlug) %}
+            a.segmentId IN (SELECT segmentId FROM segments_filtered_by_collection)
+        {% else %} a.segmentId = (SELECT segmentId FROM segments_filtered)
+        {% end %}
         {% if defined(repos) %} AND a.channel IN (SELECT channel FROM repos_to_channels) {% end %}
         AND (a.type, a.platform) IN (SELECT activityType, platform FROM activityTypes_filtered)
     ORDER BY activityType, platform

services/libs/tinybird/pipes/activity_heatmap_by_weekday_and_2hours_blocks.pipe

@@ -5,7 +5,8 @@ DESCRIPTION >
     - Returns a complete grid of all weekday/time combinations with zero counts for periods with no activity, ensuring consistent heatmap visualization.
     - Primary use case: identifying contribution patterns outside traditional work hours for community health analysis.
     - Parameters:
-    - `project`: Inherited from `segments_filtered`, project slug (e.g., 'k8s', 'tensorflow')
+    - `project`: Inherited from `segments_filtered`, project slug (e.g., 'k8s', 'tensorflow'). Required unless `collectionSlug` is given.
+    - `collectionSlug`: Inherited from `segments_filtered_by_collection`, collection slug (e.g., 'cncf'). Required unless `project` is given.
     - `repos`: Inherited from `segments_filtered`, array of repository URLs for filtering
     - `startDate`: Optional DateTime filter for activities after timestamp (e.g., '2024-01-01 00:00:00')
     - `endDate`: Optional DateTime filter for activities before timestamp (e.g., '2024-12-31 23:59:59')
@@ -18,7 +19,10 @@ SQL >
     select count(id) as activityCount, weekday, two_hours_block
     from contributions_with_local_time_ds a
     where
-        segmentId = (SELECT segmentId FROM segments_filtered)
+        {% if defined(collectionSlug) %}
+            segmentId IN (SELECT segmentId FROM segments_filtered_by_collection)
+        {% else %} segmentId = (SELECT segmentId FROM segments_filtered)
+        {% end %}
         {% if defined(startDate) %}
             AND a.timestamp
             > {{ DateTime(startDate, description="Filter activity timestamp after", required=False) }}

services/libs/tinybird/pipes/collection_contributor_dependency.pipe

@@ -0,0 +1,52 @@
+DESCRIPTION >
+    - `collection_contributor_dependency.pipe` serves the contributor dependency widget for a
+    collection's Contributors tab - identifies contributors whose combined contributions reach or
+    exceed 51% of total collection activity (bus factor analysis), aggregated across every project
+    in the collection.
+    - Collection-scoped counterpart to `contributor_dependency.pipe` - separate because that pipe
+    references `contributors_leaderboard` (project-only) by name, and the collection-scale
+    replacement for that table is `collection_contributors_leaderboard` (a differently-shaped
+    pipe - see its own DESCRIPTION for why a naive collectionSlug branch on the single-project
+    leaderboard pipe isn't used there either).
+    - Combines data from `collection_contributors_leaderboard` and `active_contributors` (both
+    already collectionSlug-aware) to provide dependency risk assessment at collection scale.
+    - Parameters:
+    - `collectionSlug`: Required collection slug (e.g., 'cncf') - inherited from `segments_filtered_by_collection`.
+    - `repos`: Optional array of repository URLs for filtering (inherited from `segments_filtered_by_collection`)
+    - `startDate`: Optional DateTime filter for activities after timestamp (e.g., '2024-01-01 00:00:00')
+    - `endDate`: Optional DateTime filter for activities before timestamp (e.g., '2024-12-31 23:59:59')
+    - `platform`: Optional string filter for source platform (e.g., 'github', 'discord', 'slack')
+    - `activity_type`: Optional string filter for single activity type (e.g., 'authored-commit')
+    - `includeCollaborations`: Optional boolean, defaults to false - same toggle as the single-project pipe.
+    - `limit`: Optional integer for the number of top contributors to rank before finding the 51%
+    cutoff. Inherited by `collection_contributors_leaderboard` (this pipe has no default of its
+    own), whose own default is 10 if omitted - callers doing bus-factor analysis should pass
+    limit=100 explicitly (same convention as the single-project `contributor_dependency.pipe` /
+    `contributors_leaderboard.pipe` pair) to reliably reach the 51% cutoff.
+    - Response: `id`, `displayName`, `githubHandleArray`, `contributionCount`, `contributionPercentage`, `roles`, `contributionPercentageRunningTotal`, `totalContributorCount`
+
+TAGS "Insights, Widget", "Collection", "Contributors"
+
+NODE collection_contributions_percentage_running_total
+SQL >
+    %
+    SELECT t.*, active_contributors.contributorCount as "totalContributorCount"
+    FROM
+        (
+            SELECT
+                id,
+                displayName,
+                githubHandleArray,
+                contributionCount,
+                contributionPercentage,
+                roles,
+                sum(contributionPercentage) OVER (
+                    ORDER BY contributionPercentage DESC, id
+                ) AS contributionPercentageRunningTotal
+            FROM collection_contributors_leaderboard
+        ) t
+    cross join active_contributors
+    WHERE
+        contributionPercentageRunningTotal <= 51
+        OR (contributionPercentageRunningTotal - contributionPercentage < 51)
+    ORDER BY contributionPercentageRunningTotal ASC

services/libs/tinybird/pipes/collection_contributors_leaderboard.pipe

@@ -0,0 +1,84 @@
+DESCRIPTION >
+    - `collection_contributors_leaderboard.pipe` serves the contributors leaderboard widget for a
+    collection's Contributors tab, aggregated across every project in the collection.
+    - Separate from `contributors_leaderboard.pipe` (the single-project version) because a naive
+    collection-scoped port of that pipe's `ROUND(COUNT(af.id) * 100.0 / SUM(COUNT(af.id)) OVER (), 2)`
+    contributionPercentage forces ClickHouse to fully materialize the per-member GROUP BY (potentially
+    hundreds of thousands of distinct members at collection scale) before it can apply ORDER BY/LIMIT -
+    confirmed via direct timing: the equivalent single-project pipe took 7-16s (once timing out
+    entirely at Tinybird's 20s limit) on a 242-project collection, vs 2-3s for the count-only path
+    with no window function. This pipe computes the total contribution count as a separate scalar
+    subquery instead of a window function, avoiding that full-materialization cost.
+    - Supports both count mode (`count=true`) and data mode for pagination-friendly leaderboard display.
+    - Always shows `displayName` (not `publicName`) since a collection spans many projects with mixed
+    LF/non-LF status - there's no single project-type answer to substitute on, unlike the single-project
+    pipe's `is_request_from_non_lf_project` check.
+    - Primary use case: powering the collection-scoped contributor ranking widget on Collection detail
+    pages' Contributors tab.
+    - Parameters:
+    - `collectionSlug`: Required collection slug (e.g., 'cncf') - inherited from `segments_filtered_by_collection`.
+    - `repos`: Optional array of repository URLs for filtering (inherited from `segments_filtered_by_collection`)
+    - `startDate`: Optional DateTime filter for activities after timestamp (e.g., '2024-01-01 00:00:00')
+    - `endDate`: Optional DateTime filter for activities before timestamp (e.g., '2024-12-31 23:59:59')
+    - `platform`: Optional string filter for source platform (e.g., 'github', 'discord', 'slack')
+    - `activity_type`: Optional string filter for single activity type (e.g., 'authored-commit')
+    - `activity_types`: Optional array of activity types (e.g., ['authored-commit', 'co-authored-commit'])
+    - `includeCollaborations`: Optional boolean, defaults to false - same toggle as the single-project pipe.
+    - `count`: Optional boolean, when true returns contributor count instead of leaderboard data
+    - `limit`: Optional integer for result pagination, defaults to 10
+    - `offset`: Optional integer for result pagination, defaults to 0
+    - Response:
+    - Count mode (`count=true`): `count` (total number of contributors)
+    - Data mode (default): `id`, `avatar`, `displayName`, `githubHandleArray`, `contributionCount`, `contributionPercentage`, `roles`
+
+TAGS "Insights, Widget", "Collection", "Contributors"
+
+NODE collection_contributors_total
+DESCRIPTION >
+    Total contribution count across the collection - a plain aggregate with no GROUP BY, so it's
+    cheap regardless of how many distinct members contributed. Computed once and reused as a scalar
+    for the percentage calculation instead of a SUM(...) OVER () window function.
+
+SQL >
+    %
+    SELECT count(af.id) AS total FROM activities_filtered af
+
+NODE collection_member_aggregates
+DESCRIPTION >
+    Per-member contribution counts, ranked and paginated. No window function here - the percentage
+    is computed in the next node against the scalar total, so ORDER BY/LIMIT can apply directly
+    after this GROUP BY without waiting on a second full-set pass.
+
+SQL >
+    %
+    {% if Boolean(count, false) %} SELECT count(distinct af.memberId) FROM activities_filtered af
+    {% else %}
+        SELECT af.memberId, count(af.id) AS contributionCount
+        FROM activities_filtered af
+        GROUP BY af.memberId
+        ORDER BY contributionCount DESC, af.memberId DESC
+        LIMIT {{ Int32(limit, 10) }}
+        OFFSET {{ Int32(offset, 0) }}
+    {% end %}
+
+NODE collection_contributors_leaderboard_result
+SQL >
+    %
+    {% if Boolean(count, false) %}
+        SELECT count(distinct af.memberId) AS count FROM activities_filtered af
+    {% else %}
+        SELECT
+            m.id,
+            m.avatar,
+            m.displayName,
+            m.githubHandleArray,
+            ma.contributionCount,
+            ROUND(ma.contributionCount * 100.0 / total.total, 2) AS contributionPercentage,
+            mr.roles
+        FROM members_sorted AS m any
+        INNER JOIN collection_member_aggregates ma ON ma.memberId = m.id
+        LEFT JOIN member_roles mr ON mr.memberId = m.id
+        CROSS JOIN collection_contributors_total total
+        WHERE m.id IN (SELECT memberId FROM collection_member_aggregates)
+        ORDER BY contributionCount DESC
+    {% end %}

services/libs/tinybird/pipes/collection_insights_aggregate.pipe

@@ -9,7 +9,11 @@ DESCRIPTION >
     - Response:
     - `projectCount`: count of distinct projects in the collection.
     - `uniqueContributorCount`: total unique contributors across all projects in the collection, deduplicated (a contributor active on multiple projects in the collection is counted once).
-    - `avgHealthScore`: average of each project's health score in the collection, rounded.
+    - `avgHealthScore`: average health score across the collection's onboarded projects, rounded. A project
+    is "onboarded" when it has at least one contributor or organization attributed to it (same definition
+    used by insightsProjects_filtered.pipe's `onboarded` filter and the frontend's project-row `isOnboarded`
+    check) — projects with no attributed contributors/organizations have no meaningful health signal yet
+    and would otherwise pull the average down artificially.
 
 TAGS "Insights, Widget", "Collection"
 
@@ -21,7 +25,7 @@ DESCRIPTION >
 
 SQL >
     %
-    SELECT id, segmentId, healthScore
+    SELECT id, segmentId, healthScore, organizationCount, contributorCount
     FROM insights_projects_populated_ds
     WHERE
         enabled = 1
@@ -38,7 +42,11 @@ SQL >
     SELECT
         (SELECT count(distinct id) FROM collection_insights_aggregate_projects) AS projectCount,
         uniq(ar.memberId) AS uniqueContributorCount,
-        (SELECT round(avg(healthScore)) FROM collection_insights_aggregate_projects) AS avgHealthScore
+        (
+            SELECT round(avg(healthScore))
+            FROM collection_insights_aggregate_projects
+            WHERE NOT (organizationCount = 0 AND contributorCount = 0)
+        ) AS avgHealthScore
     FROM activityRelations_deduplicated_cleaned_bucket_union AS ar
     WHERE
         ar.memberId != ''