Skip to content

Commit 9d1ab68

Browse files
committed
fix: filter out bad timestamp issue activities to prevent overflows
Signed-off-by: anil <epipav@gmail.com>
1 parent 5b6ba59 commit 9d1ab68

1 file changed

Lines changed: 16 additions & 4 deletions

File tree

services/libs/tinybird/pipes/issue_analysis_copy_pipe.pipe

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
DESCRIPTION >
2-
Compacts activities from same issue into one, keeping necessary information in a single row. Helps to serve issue-wide widgets in the development tab.
2+
Compacts activities from same issue into one, keeping necessary information in a single row. Helps to serve issue-wide widgets in the development tab.
3+
34

45
NODE issues_opened
56
SQL >
@@ -13,24 +14,33 @@ SQL >
1314
memberId,
1415
organizationId
1516
FROM activityRelations_deduplicated_cleaned_bucket_union
16-
WHERE type = 'issues-opened'
17+
WHERE type = 'issues-opened' AND toYear(timestamp) >= 1971
18+
19+
1720

1821
NODE issues_closed
1922
SQL >
23+
2024
SELECT sourceParentId, MIN(timestamp) AS closedAt
2125
FROM activityRelations_deduplicated_cleaned_bucket_union
22-
WHERE type = 'issues-closed' AND sourceParentId != ''
26+
WHERE type = 'issues-closed' AND sourceParentId != '' AND toYear(timestamp) >= 1971
2327
GROUP BY sourceParentId
2428

29+
30+
2531
NODE issues_comment
2632
SQL >
33+
2734
SELECT sourceParentId, MIN(timestamp) AS commentedAt
2835
FROM activityRelations_deduplicated_cleaned_bucket_union
2936
WHERE type = 'issue-comment' AND sourceParentId != '' AND toYear(timestamp) >= 1971
3037
GROUP BY sourceParentId
3138

39+
40+
3241
NODE issue_analysis_results_merged
3342
SQL >
43+
3444
SELECT
3545
opened.id,
3646
opened.sourceId,
@@ -57,7 +67,9 @@ SQL >
5767
LEFT JOIN issues_closed AS closed ON opened.sourceId = closed.sourceParentId
5868
LEFT JOIN issues_comment AS comment ON opened.sourceId = comment.sourceParentId
5969

60-
TYPE COPY
70+
TYPE copy
6171
TARGET_DATASOURCE issues_analyzed
6272
COPY_MODE replace
6373
COPY_SCHEDULE 20 * * * *
74+
75+

0 commit comments

Comments
 (0)