Skip to content

fix: catchup shows no codes after startup backfill#38

Merged
BigMichi1 merged 1 commit into
mainfrom
fix/catchup-no-codes-available
May 22, 2026
Merged

fix: catchup shows no codes after startup backfill#38
BigMichi1 merged 1 commit into
mainfrom
fix/catchup-no-codes-available

Conversation

@BigMichi1
Copy link
Copy Markdown
Owner

Problem

Users who registered via /setup after the bot started were seeing "ℹ️ No Codes Available" when running /catchup, even though the startup backfill had found codes.

Three separate bugs caused this:


Bug 1 — Pending codes were never saved to the database

backfillHandler.ts incremented stats.pendingCodes but never called codeManager.addPendingCode(). The codes existed only in memory, so /catchup (which reads from the database) found nothing.

Fix: Added await codeManager.addPendingCode(code) in the trailing loop that processes unredeemed codes.


Bug 2 — All user redemptions during backfill were silently skipped

backfillHandler.ts checked 'success' in userDetailsResponse to validate the API response. Valid PlayerData objects never have a success property (they have details), so the condition was always false. Every user's redemption attempt was logged as "not a valid response" and skipped.

Fix: Changed the check to !playerData?.details.


Bug 3 — Multiple missing codeManager methods

Several methods called by production commands and tests were never implemented:

  • normalizeCodeStatus — converts numeric API status codes to canonical strings
  • isCodeRedeemedByUser — checks if a specific user redeemed a code
  • getAllValidCodes — used by /catchup to find redeemable codes
  • addNewPendingCodes — bulk insert, returns only newly added codes
  • getRedeemedCodeCount — count of redemptions for a user
  • deleteUserRedeemedCodes — used by /deleteaccount
  • getRedeemedCodesByUsers — bulk lookup for the auto-redeemer
  • getServerCodeStats — used by /stats
  • getAggregateLoot — now backed by the loot_totals table
  • backfillLootTotals — one-time migration to seed loot_totals

Also fixed:

  • addRedeemedCode: status normalization, correct onConflictDoUpdate target ([code, discordId]), proper isPublic propagation, writes loot to loot_totals on Success
  • addPendingCode: added onConflictDoNothing() to make it idempotent
  • getRedeemedCodeDetails: added optional offset parameter

Testing

Added src/bot/handlers/backfillHandler.test.ts with 6 end-to-end tests covering:

  • Startup backfill with no registered users → codes stored as pending → new user /catchup works
  • API failure during backfill falls back to storing codes as pending
  • Duplicate backfill runs are idempotent

All 270 tests across 10 files pass.

Copilot AI review requested due to automatic review settings May 22, 2026 17:05
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes /catchup returning “No Codes Available” after startup backfill by ensuring backfill persists pending codes, correctly validates getUserDetails responses, and fills in missing/incorrect codeManager behaviors relied on by commands and background jobs.

Changes:

  • Persist unredeemed backfill-discovered codes into pending_codes so /catchup can redeem them later.
  • Correct backfill API response validation to detect valid PlayerData responses via the details shape.
  • Add/adjust codeManager methods (status normalization, pending/redeemed lookups, valid-code listing, loot totals backfill) and add E2E backfill regression tests.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/bot/handlers/backfillHandler.ts Fixes backfill user-details validation and persists unredeemed codes as pending.
src/bot/handlers/backfillHandler.test.ts Adds end-to-end regression tests covering startup backfill → setup → catchup and idempotency.
src/bot/database/codeManager.ts Implements/adjusts code status normalization, pending/redeemed queries, bulk operations, and loot totals logic.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/bot/database/codeManager.ts Outdated
Comment thread src/bot/database/codeManager.ts Outdated
Comment thread src/bot/database/codeManager.ts
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 22, 2026

✅ Coverage Report

Metric Value
Current coverage 70.53% (1544/2189)
Baseline coverage 68.75% (1355/1971)
Result PASS

New Files

File Coverage Lines Status
src/bot/handlers/backfillHandler.test.ts SKIP (ignored)

Changed Files

File Baseline Current Status
src/bot/database/codeManager.test.ts SKIP (ignored)
src/bot/database/codeManager.ts 100.00% 100.00% PASS
src/bot/handlers/backfillHandler.ts N/A N/A SKIP (new to coverage)

Copilot AI review requested due to automatic review settings May 22, 2026 17:17
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

Comment thread src/bot/handlers/backfillHandler.ts
Comment thread src/bot/handlers/backfillHandler.ts Outdated
Comment thread src/bot/database/codeManager.ts Outdated
Comment thread src/bot/database/codeManager.ts
BigMichi1 added a commit that referenced this pull request May 22, 2026
normalizeCodeStatus: replace parseInt with /^\d+$/ guard so partial
numeric strings like '4foo' are no longer misclassified as a known
status code.

deleteUserRedeemedCodes: replace RETURNING-all-ids approach with a
COUNT(*) + DELETE pair to avoid materialising a large array of row IDs
for users with many redemption records.

backfillHandler: validate instance_id is present and non-'0' after
confirming playerData.details exists, matching the check in
autoRedeemer.ts. Skip the submitCode call when instance_id is invalid.

backfillHandler: replace the N+1 isCodeRedeemed/addPendingCode loop
with a single getRedeemedCodesFromList query to build the excluded set,
then bulk-insert the remainder via addNewPendingCodes. Adds
getRedeemedCodesFromList to codeManager (SELECT … IN with Set return).

Tests: add normalizeCodeStatus partial-string case, getRedeemedCodesFromList
unit tests, and a backfillHandler scenario covering invalid instance_id.

Signed-off-by: Michael Cramer <michael@bigmichi1.de>
Copilot AI review requested due to automatic review settings May 22, 2026 17:40
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

src/bot/handlers/backfillHandler.ts:252

  • stats.pendingCodes is set to inserted.length, which only counts newly-inserted pending codes. If some codes were already present in pending_codes (e.g., repeated backfills), the progress message will under-report how many codes are pending/available. If the intent is “pending codes found this run”, consider using codesToPend.length; if the intent is “newly inserted”, consider adjusting the label/message accordingly.
    const redeemedSet = await codeManager.getRedeemedCodesFromList(allCodesArr);
    const codesToPend = allCodesArr.filter((c) => !redeemedSet.has(c));
    const inserted = await codeManager.addNewPendingCodes(codesToPend);
    stats.pendingCodes = inserted.length;

    onProgress?.(
      `✅ Backfill complete! Found: ${stats.codesFound}, Redeemed: ${stats.codesRedeemed}, Pending: ${stats.pendingCodes}`
    );

Comment thread src/bot/database/codeManager.ts Outdated
Comment thread src/bot/database/codeManager.ts
Comment thread src/bot/database/codeManager.ts
Copilot AI review requested due to automatic review settings May 22, 2026 17:47
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Comment thread src/bot/handlers/backfillHandler.ts
Comment thread src/bot/database/codeManager.ts
Copilot AI review requested due to automatic review settings May 22, 2026 17:52
@BigMichi1 BigMichi1 force-pushed the fix/catchup-no-codes-available branch from 7780e44 to 3d9af10 Compare May 22, 2026 17:56
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

Comment thread src/bot/handlers/backfillHandler.ts Outdated
Comment thread src/bot/handlers/backfillHandler.ts Outdated
Comment thread src/bot/database/codeManager.ts
Comment thread src/bot/database/codeManager.ts Outdated
BigMichi1 added a commit that referenced this pull request May 22, 2026
backfillHandler: normalize instance_id with String(...).trim() before
comparing to '0', matching the coercion used in catchup.ts and preventing
numbers or whitespace-padded values from bypassing the guard.

backfillHandler: use inserted.length (return value of addNewPendingCodes)
for stats.pendingCodes instead of codesToPend.length, so re-runs do not
over-report codes as newly pending when they already existed.

getPublicUnexpiredCodes: deduplicate by code in memory (keep the latest
row, already first due to ORDER BY redeemedAt DESC) so that the same code
redeemed by multiple users does not appear more than once in the result.
Adds a test covering the multi-user same-code case.

getAllValidCodes: replace the two-query JS-filter approach with a single
SQL query using a NOT IN subquery on the Code Expired set, letting SQLite
handle the exclusion with index support instead of pulling both lists into
JS. Adds notInArray to the drizzle-orm import.

Signed-off-by: Michael Cramer <michael@bigmichi1.de>
Three bugs prevented /catchup from showing codes to newly registered users:

1. backfillHandler never persisted pending codes - incremented stats.pendingCodes
   counter but never called codeManager.addPendingCode(), so no codes were
   stored in the database for /catchup to find.

2. backfillHandler skipped all user redemptions - checked 'success' in
   userDetailsResponse which is never true for valid PlayerData responses
   (they have a 'details' property, not 'success'). Fixed by checking
   !playerData?.details instead.

3. codeManager was missing several methods used by production code and tests:
   normalizeCodeStatus, isCodeRedeemedByUser, getAllValidCodes,
   addNewPendingCodes, getRedeemedCodeCount, deleteUserRedeemedCodes,
   getRedeemedCodesByUsers, getServerCodeStats, getAggregateLoot (now backed
   by loot_totals table), and backfillLootTotals.

Additional fixes in codeManager:
- addRedeemedCode: normalize numeric status codes via /^\d+$/ guard,
  fix onConflictDoUpdate target to [code, discordId], propagate isPublic
  flag correctly, write loot aggregates to loot_totals on Success.
- addPendingCode: add onConflictDoNothing() to make it idempotent.
- getRedeemedCodeDetails: add optional offset parameter.
- addNewPendingCodes: chunk inserts at 499 rows (2 params/row) to stay
  within SQLite 999-variable limit.
- getRedeemedCodesFromList: chunk IN queries at 997 codes (+2 status
  params) to stay within SQLite 999-variable limit.
- deleteUserRedeemedCodes: COUNT(*) + DELETE instead of RETURNING to
  avoid materialising a large deleted-row array.
- getRedeemedCodesByUsers: chunk discordIds within per-query param budget.
- getPublicUnexpiredCodes: deduplicate by code (keep latest per code).
- getAllValidCodes: single SQL query using NOT IN subquery on expired set.
- Export CHEST_TYPE_NAMES and LootSummary type (used by codes.ts/stats.ts).

backfillHandler fixes:
- Normalize instance_id with String(...).trim() before comparing to '0'.
- Use inserted.length from addNewPendingCodes() for stats.pendingCodes so
  re-runs do not over-report codes as newly pending.
- Validate instance_id is present and non-'0' before calling submitCode,
  matching the guard in autoRedeemer.ts.
- Replace N+1 isCodeRedeemed/addPendingCode loop with a single
  getRedeemedCodesFromList query + bulk addNewPendingCodes insert.

Adds end-to-end tests covering the startup-backfill -> setup -> catchup
flow, including API failure fallback, duplicate backfill idempotency,
invalid instance_id skip, SQLite chunking boundary paths, and
getPublicUnexpiredCodes multi-user deduplication.

Signed-off-by: Michael Cramer <michael@bigmichi1.de>
@BigMichi1 BigMichi1 force-pushed the fix/catchup-no-codes-available branch from be5090d to 2ef7cbb Compare May 22, 2026 18:12
@BigMichi1 BigMichi1 merged commit c463b35 into main May 22, 2026
12 checks passed
@BigMichi1 BigMichi1 deleted the fix/catchup-no-codes-available branch May 22, 2026 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants