perf(worker): Consolidate flake updates in test analytics

[sentry]

Fixes WORKER-Y9A. The issue was that: Repeatedly fetching all repository flakes for each commit processed in a batch causes an N+1 query.

• Moved fetching of current flakes to process_flakes_for_repo to fetch once per repository.
• Modified process_flakes_for_commit to accept and update a shared curr_flakes dictionary.
• Consolidated the bulk creation/update of Flake objects to occur once per repository after all commits are processed.
• This change significantly reduces database calls and improves performance for test analytics flake processing.

This fix was generated by Seer in Sentry, triggered automatically. 👁️ Run ID: 13593210

Not quite right? Click here to continue debugging with Seer.

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

---

Note

Low Risk
Low risk performance refactor that mainly changes when/where flakes are fetched and bulk-written; primary concern is ensuring deferred Flake.objects.bulk_create still persists all intended updates after multi-commit processing.

Overview
Reduces N+1 DB work in test-analytics flake processing by fetching curr_flakes once per repo in process_flakes_for_repo and passing the shared dict into process_flakes_for_commit for all commits in the batch.

Defers the Flake.objects.bulk_create(... update_conflicts=True ...) upsert to run once after all queued commits are processed, and forces commit uploads evaluation via list(get_relevant_uploads(...)) to support logging/iteration without re-querying.

^{Reviewed by Cursor Bugbot for commit 6fff16b. Bugbot is set up for automated code reviews on this repo. Configure here.}

[codecov-notifications]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

[sentry]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.26%. Comparing base (548e0c1) to head (6fff16b).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #879   +/-   ##
=======================================
  Coverage   92.26%   92.26%           
=======================================
  Files        1307     1307           
  Lines       48011    48011           
  Branches     1632     1632           
=======================================
  Hits        44295    44295           
  Misses       3407     3407           
  Partials      309      309           

Flag	Coverage Δ
workerintegration	58.54% <14.28%> (ø)
workerunit	90.39% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 6fff16b. Configure here.}

[cursor]

Deferred bulk_create loses all flake data on mid-processing error

Medium Severity

Moving bulk_create to after the while loop means that if any exception occurs during processing (e.g., a DB error in Testrun.objects.bulk_update or get_relevant_uploads), ALL accumulated flake changes across ALL lpop batches are lost. Meanwhile, testrun outcomes are already persisted per-upload via bulk_update in process_single_upload, and commit IDs are already removed from Redis by lpop — so they won't be reprocessed. Previously, flakes were persisted after each commit, so only the failing commit's data was at risk. The sibling implementation in detect_flakes.py handles this correctly by calling bulk_create per iteration inside transaction.atomic().

Additional Locations (1)

• apps/worker/services/test_analytics/ta_process_flakes.py#L139-L143

 

^{Reviewed by Cursor Bugbot for commit 6fff16b. Configure here.}