fix(indexstore): make UpsertRun overwrite existing rows and stop dropping recent runs#253
Merged
skylenet merged 3 commits intoJul 1, 2026
Conversation
… order deterministic The UpsertRun ON CONFLICT clause used UpdateAll, which overwrote indexed_at with the re-index time on every re-index, losing the original first-index timestamp (exposed via the API as indexed_at, distinct from reindexed_at). Replace UpdateAll with an explicit column list that omits id and indexed_at, so zero-valued fields are still written but indexed_at is preserved. Also add a secondary sort key (run_id DESC) to ListTestStatsBySuiteRecent so that when more runs than the per-client cap share the same run_start, the same runs are kept deterministically on every call and across SQLite and Postgres. Add a regression test asserting indexed_at survives a re-index while reindexed_at is recorded.
Member
|
Thank you so much for your fixes @damilolaedwards 🙏 |
…rt-and-recent-runs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two data-correctness fixes in the index store.
UpsertRun never updated existing rows
UpsertRunranWhere(...).Assign(run).FirstOrCreate(run)with the same struct pointer as both the assign source and the query destination.FirstOrCreateloads the found row intorunbefore building the update, so the update set every column to the value it already had, a no-op. Re-indexing a run (status change, corrected test counts,HasResultflip, end timestamp) silently kept the original values, and zero-valued fields could never be written.Replaced it with an
INSERT ... ON CONFLICT(discovery_path, run_id) DO UPDATEupsert that writes every column, including zero values, and still does not duplicate the row.ListTestStatsBySuiteRecent dropped recent runs
The query used
SELECT DISTINCT client, run_id, run_startand counted per row. A run whose stats carry inconsistentrun_startvalues (e.g. a backfill rewrote some) produced several rows for onerun_id, consumed several of the per-client slots, and evicted other recent runs. Changed it toSELECT client, run_id, MAX(run_start) ... GROUP BY client, run_idso each run counts once.Tests
upsert_recent_test.go: re-index overwrites all fields including zeros without duplicating; insert path; inconsistentrun_startreturns both runs; per-client cap still honored.TestStore_UpsertRunIdempotent, which previously asserted the buggy first-write-wins behavior, to expect last-write-wins.Full
pkg/api/indexstoresuite passes;go vetand gofmt clean.