fix: catchup shows no codes after startup backfill#38
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
Fixes /catchup returning “No Codes Available” after startup backfill by ensuring backfill persists pending codes, correctly validates getUserDetails responses, and fills in missing/incorrect codeManager behaviors relied on by commands and background jobs.
Changes:
- Persist unredeemed backfill-discovered codes into
pending_codesso/catchupcan redeem them later. - Correct backfill API response validation to detect valid
PlayerDataresponses via thedetailsshape. - Add/adjust
codeManagermethods (status normalization, pending/redeemed lookups, valid-code listing, loot totals backfill) and add E2E backfill regression tests.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| src/bot/handlers/backfillHandler.ts | Fixes backfill user-details validation and persists unredeemed codes as pending. |
| src/bot/handlers/backfillHandler.test.ts | Adds end-to-end regression tests covering startup backfill → setup → catchup and idempotency. |
| src/bot/database/codeManager.ts | Implements/adjusts code status normalization, pending/redeemed queries, bulk operations, and loot totals logic. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
✅ Coverage Report
New Files
Changed Files
|
BigMichi1
added a commit
that referenced
this pull request
May 22, 2026
normalizeCodeStatus: replace parseInt with /^\d+$/ guard so partial numeric strings like '4foo' are no longer misclassified as a known status code. deleteUserRedeemedCodes: replace RETURNING-all-ids approach with a COUNT(*) + DELETE pair to avoid materialising a large array of row IDs for users with many redemption records. backfillHandler: validate instance_id is present and non-'0' after confirming playerData.details exists, matching the check in autoRedeemer.ts. Skip the submitCode call when instance_id is invalid. backfillHandler: replace the N+1 isCodeRedeemed/addPendingCode loop with a single getRedeemedCodesFromList query to build the excluded set, then bulk-insert the remainder via addNewPendingCodes. Adds getRedeemedCodesFromList to codeManager (SELECT … IN with Set return). Tests: add normalizeCodeStatus partial-string case, getRedeemedCodesFromList unit tests, and a backfillHandler scenario covering invalid instance_id. Signed-off-by: Michael Cramer <michael@bigmichi1.de>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (1)
src/bot/handlers/backfillHandler.ts:252
stats.pendingCodesis set toinserted.length, which only counts newly-inserted pending codes. If some codes were already present inpending_codes(e.g., repeated backfills), the progress message will under-report how many codes are pending/available. If the intent is “pending codes found this run”, consider usingcodesToPend.length; if the intent is “newly inserted”, consider adjusting the label/message accordingly.
const redeemedSet = await codeManager.getRedeemedCodesFromList(allCodesArr);
const codesToPend = allCodesArr.filter((c) => !redeemedSet.has(c));
const inserted = await codeManager.addNewPendingCodes(codesToPend);
stats.pendingCodes = inserted.length;
onProgress?.(
`✅ Backfill complete! Found: ${stats.codesFound}, Redeemed: ${stats.codesRedeemed}, Pending: ${stats.pendingCodes}`
);
7780e44 to
3d9af10
Compare
BigMichi1
added a commit
that referenced
this pull request
May 22, 2026
backfillHandler: normalize instance_id with String(...).trim() before comparing to '0', matching the coercion used in catchup.ts and preventing numbers or whitespace-padded values from bypassing the guard. backfillHandler: use inserted.length (return value of addNewPendingCodes) for stats.pendingCodes instead of codesToPend.length, so re-runs do not over-report codes as newly pending when they already existed. getPublicUnexpiredCodes: deduplicate by code in memory (keep the latest row, already first due to ORDER BY redeemedAt DESC) so that the same code redeemed by multiple users does not appear more than once in the result. Adds a test covering the multi-user same-code case. getAllValidCodes: replace the two-query JS-filter approach with a single SQL query using a NOT IN subquery on the Code Expired set, letting SQLite handle the exclusion with index support instead of pulling both lists into JS. Adds notInArray to the drizzle-orm import. Signed-off-by: Michael Cramer <michael@bigmichi1.de>
Three bugs prevented /catchup from showing codes to newly registered users: 1. backfillHandler never persisted pending codes - incremented stats.pendingCodes counter but never called codeManager.addPendingCode(), so no codes were stored in the database for /catchup to find. 2. backfillHandler skipped all user redemptions - checked 'success' in userDetailsResponse which is never true for valid PlayerData responses (they have a 'details' property, not 'success'). Fixed by checking !playerData?.details instead. 3. codeManager was missing several methods used by production code and tests: normalizeCodeStatus, isCodeRedeemedByUser, getAllValidCodes, addNewPendingCodes, getRedeemedCodeCount, deleteUserRedeemedCodes, getRedeemedCodesByUsers, getServerCodeStats, getAggregateLoot (now backed by loot_totals table), and backfillLootTotals. Additional fixes in codeManager: - addRedeemedCode: normalize numeric status codes via /^\d+$/ guard, fix onConflictDoUpdate target to [code, discordId], propagate isPublic flag correctly, write loot aggregates to loot_totals on Success. - addPendingCode: add onConflictDoNothing() to make it idempotent. - getRedeemedCodeDetails: add optional offset parameter. - addNewPendingCodes: chunk inserts at 499 rows (2 params/row) to stay within SQLite 999-variable limit. - getRedeemedCodesFromList: chunk IN queries at 997 codes (+2 status params) to stay within SQLite 999-variable limit. - deleteUserRedeemedCodes: COUNT(*) + DELETE instead of RETURNING to avoid materialising a large deleted-row array. - getRedeemedCodesByUsers: chunk discordIds within per-query param budget. - getPublicUnexpiredCodes: deduplicate by code (keep latest per code). - getAllValidCodes: single SQL query using NOT IN subquery on expired set. - Export CHEST_TYPE_NAMES and LootSummary type (used by codes.ts/stats.ts). backfillHandler fixes: - Normalize instance_id with String(...).trim() before comparing to '0'. - Use inserted.length from addNewPendingCodes() for stats.pendingCodes so re-runs do not over-report codes as newly pending. - Validate instance_id is present and non-'0' before calling submitCode, matching the guard in autoRedeemer.ts. - Replace N+1 isCodeRedeemed/addPendingCode loop with a single getRedeemedCodesFromList query + bulk addNewPendingCodes insert. Adds end-to-end tests covering the startup-backfill -> setup -> catchup flow, including API failure fallback, duplicate backfill idempotency, invalid instance_id skip, SQLite chunking boundary paths, and getPublicUnexpiredCodes multi-user deduplication. Signed-off-by: Michael Cramer <michael@bigmichi1.de>
be5090d to
2ef7cbb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Users who registered via
/setupafter the bot started were seeing "ℹ️ No Codes Available" when running/catchup, even though the startup backfill had found codes.Three separate bugs caused this:
Bug 1 — Pending codes were never saved to the database
backfillHandler.tsincrementedstats.pendingCodesbut never calledcodeManager.addPendingCode(). The codes existed only in memory, so/catchup(which reads from the database) found nothing.Fix: Added
await codeManager.addPendingCode(code)in the trailing loop that processes unredeemed codes.Bug 2 — All user redemptions during backfill were silently skipped
backfillHandler.tschecked'success' in userDetailsResponseto validate the API response. ValidPlayerDataobjects never have asuccessproperty (they havedetails), so the condition was always false. Every user's redemption attempt was logged as "not a valid response" and skipped.Fix: Changed the check to
!playerData?.details.Bug 3 — Multiple missing
codeManagermethodsSeveral methods called by production commands and tests were never implemented:
normalizeCodeStatus— converts numeric API status codes to canonical stringsisCodeRedeemedByUser— checks if a specific user redeemed a codegetAllValidCodes— used by/catchupto find redeemable codesaddNewPendingCodes— bulk insert, returns only newly added codesgetRedeemedCodeCount— count of redemptions for a userdeleteUserRedeemedCodes— used by/deleteaccountgetRedeemedCodesByUsers— bulk lookup for the auto-redeemergetServerCodeStats— used by/statsgetAggregateLoot— now backed by theloot_totalstablebackfillLootTotals— one-time migration to seedloot_totalsAlso fixed:
addRedeemedCode: status normalization, correctonConflictDoUpdatetarget ([code, discordId]), properisPublicpropagation, writes loot toloot_totalson SuccessaddPendingCode: addedonConflictDoNothing()to make it idempotentgetRedeemedCodeDetails: added optionaloffsetparameterTesting
Added
src/bot/handlers/backfillHandler.test.tswith 6 end-to-end tests covering:/catchupworksAll 270 tests across 10 files pass.