Skip to content

Commit 04e01e0

Browse files
committed
feat: add message history backfill feature with rate limiting
- Add /backfill command for recovering missed codes from message history - Implement automatic startup backfill (runs if last backfill >6 hours ago) - Add global lock to prevent concurrent backfill operations - Implement per-user rate limiting (1 hour cooldown between initiations) - Add backfill_operations table to track all backfill runs - Require 'Manage Messages' permission for /backfill command - Fetch user history in 100-message batches respecting Discord API limits - Auto-redeem found codes for users with stored credentials - Add comprehensive backfill documentation in docs/README.md and DEVELOPMENT.md
1 parent f55d9e5 commit 04e01e0

9 files changed

Lines changed: 581 additions & 4 deletions

File tree

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,8 +63,9 @@ See [docker-compose.example.yml](docker-compose.example.yml) for all available c
6363

6464
## ✨ Features
6565

66-
- 🤖 **Slash Commands** - `/setup`, `/redeem`, `/inventory`, `/open`, `/blacksmith`, `/codes`, `/makepublic`, `/help`
66+
- 🤖 **Slash Commands** - `/setup`, `/redeem`, `/inventory`, `/open`, `/blacksmith`, `/codes`, `/makepublic`, `/backfill`, `/help`
6767
- 🔄 **Auto Code Detection** - Scans Discord messages for codes automatically
68+
- ⏮️ **Message History Backfill** - Recover missed codes from message history with built-in rate limiting
6869
- 🎁 **Code Redemption** - Submit codes and get rewards
6970
- 📦 **Chest Management** - Open chests and view loot
7071
- ⚒️ **Blacksmith** - Upgrade heroes with contracts

docs/DEVELOPMENT.md

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,13 +40,18 @@ src/bot/
4040
│ ├── inventory.ts # Show account info
4141
│ ├── open.ts # Open chests
4242
│ ├── blacksmith.ts # Upgrade heroes
43+
│ ├── codes.ts # Show code history
44+
│ ├── makepublic.ts # Share codes with other users
45+
│ ├── backfill.ts # Recover missed codes from history
4346
│ └── help.ts # Command help
4447
├── database/
4548
│ ├── db.ts # SQLite connection & queries
4649
│ ├── userManager.ts # User credentials storage
47-
│ └── codeManager.ts # Code tracking & history
50+
│ ├── codeManager.ts # Code tracking & history
51+
│ └── backfillManager.ts # Backfill operations & locking
4852
├── handlers/
49-
│ └── codeScanner.ts # Message code detection
53+
│ ├── codeScanner.ts # Message code detection
54+
│ └── backfillHandler.ts # Message history scanning & redemption
5055
└── utils/
5156
└── debugLogger.ts # Response logging & cleanup
5257
@@ -155,6 +160,14 @@ SQLite database (`./data/idle.db`) contains:
155160
- details (JSON)
156161
- timestamp
157162

163+
**backfill_operations**
164+
165+
- id (PK)
166+
- initiated_by (user who initiated or "system" for automatic)
167+
- started_at, completed_at
168+
- codes_found, codes_redeemed (counts)
169+
- status (in_progress, completed, failed)
170+
158171
## Testing Commands
159172

160173
```

docs/README.md

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,9 @@ A Discord bot that automatically scans for and redeems Idle Champions promo code
44

55
## Features
66

7-
- 🤖 **Slash Commands** - `/setup`, `/redeem`, `/inventory`, `/open`, `/blacksmith`, `/codes`, `/makepublic`, `/help`
7+
- 🤖 **Slash Commands** - `/setup`, `/redeem`, `/inventory`, `/open`, `/blacksmith`, `/codes`, `/makepublic`, `/backfill`, `/help`
88
- 🔄 **Auto Code Detection** - Scans Discord messages for codes automatically
9+
- ⏮️ **Message History Backfill** - Recover missed codes from message history (protected with rate limiting)
910
- 🎁 **Code Redemption** - Submit codes and get rewards
1011
- 📦 **Chest Management** - Open chests and view loot
1112
- ⚒️ **Blacksmith** - Upgrade heroes with contracts
@@ -62,6 +63,7 @@ brew install mise
6263
| `/blacksmith contract_type:<type> hero_id:<id> count:<count>` | Upgrade heroes |
6364
| `/codes [count:<count>]` | Show your redeemed codes history (last 10) |
6465
| `/makepublic code:<code>` | Share one of your redeemed codes with other users |
66+
| `/backfill [channel:<channel>]` | Recover missed codes from message history |
6567
| `/help` | Show all commands |
6668

6769
### Setup & Authentication
@@ -142,6 +144,28 @@ Share one of your previously redeemed codes with other users. Other users can re
142144
- **Note:** Codes automatically become public when a second user successfully redeems them
143145
- **Example:** `/makepublic code:SHARED123`
144146

147+
#### `/backfill [channel:<channel>]`
148+
149+
Recover missed codes from Discord message history. Scans the entire message history of a channel and redeems any codes that weren't previously found.
150+
151+
- **Optional parameters:**
152+
- `channel` - Target channel to scan (defaults to current channel)
153+
- **Requirements:**
154+
- You must have "Manage Messages" permission (admin-only)
155+
- Only works on text channels
156+
- **Protection:**
157+
- Only one backfill can run at a time (global lock)
158+
- Per-user rate limit: 1 hour between initiations
159+
- Automatically runs on startup if last run was >6 hours ago
160+
- **Response:** Shows stats (codes found, redeemed, pending, any errors)
161+
- **Example:** `/backfill` or `/backfill channel:#code-drops`
162+
163+
**Automatic Startup Backfill:**
164+
- Runs automatically when the bot starts if `DISCORD_CHANNEL_ID` is configured
165+
- Only runs if the last backfill was more than 6 hours ago
166+
- Helps catch codes that appeared while the bot was offline
167+
- No manual action needed
168+
145169
### Help
146170

147171
#### `/help`

src/bot/bot.ts

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,10 @@ import dotenv from 'dotenv';
22
import { Client, Collection, Events, GatewayIntentBits, MessageFlags } from 'discord.js';
33
import { db } from './database/db';
44
import { scanMessageForCodes } from './handlers/codeScanner';
5+
import { backfillChannelHistory } from './handlers/backfillHandler';
56
import { codeManager } from './database/codeManager';
67
import { userManager } from './database/userManager';
8+
import { backfillManager } from './database/backfillManager';
79
import IdleChampionsApi from './api/idleChampionsApi';
810
import { initDebugLogger } from './utils/debugLogger';
911
import logger from './utils/logger';
@@ -71,6 +73,42 @@ client.on(Events.ClientReady, async () => {
7173
await db.initialize();
7274
logger.info('Database initialized');
7375

76+
// Check if startup backfill should run
77+
const shouldBackfill = await backfillManager.shouldRunStartupBackfill();
78+
if (shouldBackfill && CHANNEL_ID) {
79+
logger.info('Running startup backfill...');
80+
try {
81+
const channel = await client.channels.fetch(CHANNEL_ID);
82+
if (channel) {
83+
const operationId = await backfillManager.startBackfill(client.user?.id || 'system');
84+
85+
const stats = await backfillChannelHistory(channel, (message) => {
86+
logger.info(`[STARTUP BACKFILL] ${message}`);
87+
});
88+
89+
await backfillManager.updateBackfill(
90+
operationId,
91+
stats.codesFound,
92+
stats.codesRedeemed,
93+
stats.errors.length === 0 ? 'completed' : 'failed'
94+
);
95+
96+
logger.info(
97+
`Startup backfill completed: found=${stats.codesFound}, redeemed=${stats.codesRedeemed}`
98+
);
99+
} else {
100+
logger.warn('Startup backfill: Could not fetch channel');
101+
}
102+
} catch (backfillError) {
103+
logger.error('Error during startup backfill:', backfillError);
104+
}
105+
} else if (!shouldBackfill) {
106+
const lastBackfill = await backfillManager.getLastBackfill();
107+
logger.info(`Skipping startup backfill - last run was less than 6 hours ago at ${lastBackfill?.completed_at}`);
108+
} else {
109+
logger.info('Skipping startup backfill - DISCORD_CHANNEL_ID not configured');
110+
}
111+
74112
// Register slash commands
75113
logger.debug(`GUILD_ID from env: ${GUILD_ID}`);
76114
if (GUILD_ID) {

src/bot/commands/backfill.ts

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
import {
2+
SlashCommandBuilder,
3+
ChatInputCommandInteraction,
4+
EmbedBuilder,
5+
MessageFlags,
6+
PermissionFlagsBits,
7+
ChannelType,
8+
} from 'discord.js';
9+
import { backfillManager } from '../database/backfillManager';
10+
import { backfillChannelHistory } from '../handlers/backfillHandler';
11+
import logger from '../utils/logger';
12+
13+
export const data = new SlashCommandBuilder()
14+
.setName('backfill')
15+
.setDescription('Backfill message history to find and redeem missed codes')
16+
.setDefaultMemberPermissions(PermissionFlagsBits.ManageMessages)
17+
.addChannelOption((option) =>
18+
option
19+
.setName('channel')
20+
.setDescription('The channel to backfill (defaults to current channel)')
21+
.setRequired(false)
22+
);
23+
24+
export async function execute(interaction: ChatInputCommandInteraction) {
25+
try {
26+
logger.info(`[BACKFILL CMD] Started by ${interaction.user.tag}`);
27+
28+
// Check permissions
29+
if (!interaction.memberPermissions?.has(PermissionFlagsBits.ManageMessages)) {
30+
const embed = new EmbedBuilder()
31+
.setColor(0xff0000)
32+
.setTitle('❌ Permission Denied')
33+
.setDescription('You need the "Manage Messages" permission to run backfill.');
34+
35+
await interaction.reply({ embeds: [embed], flags: MessageFlags.Ephemeral });
36+
return;
37+
}
38+
39+
// Check if backfill is already in progress
40+
if (backfillManager.isBackfillInProgress()) {
41+
const embed = new EmbedBuilder()
42+
.setColor(0xffaa00)
43+
.setTitle('⚠️ Backfill In Progress')
44+
.setDescription(
45+
'A backfill operation is already running. Please wait for it to complete.'
46+
);
47+
48+
await interaction.reply({ embeds: [embed], flags: MessageFlags.Ephemeral });
49+
return;
50+
}
51+
52+
// Check rate limiting
53+
const canInitiate = await backfillManager.canUserInitiateBackfill(interaction.user.id);
54+
if (!canInitiate) {
55+
const embed = new EmbedBuilder()
56+
.setColor(0xffaa00)
57+
.setTitle('⏱️ Rate Limited')
58+
.setDescription(
59+
'You can only initiate a backfill once per hour. Please try again later.'
60+
);
61+
62+
await interaction.reply({ embeds: [embed], flags: MessageFlags.Ephemeral });
63+
return;
64+
}
65+
66+
// Get the target channel
67+
let targetChannel: any = interaction.options.getChannel('channel');
68+
if (!targetChannel) {
69+
targetChannel = interaction.channel;
70+
}
71+
72+
if (!targetChannel || targetChannel.type !== ChannelType.GuildText) {
73+
const embed = new EmbedBuilder()
74+
.setColor(0xff0000)
75+
.setTitle('❌ Error')
76+
.setDescription('Invalid channel - must be a text channel in this server.');
77+
78+
await interaction.reply({ embeds: [embed], flags: MessageFlags.Ephemeral });
79+
return;
80+
}
81+
82+
// Defer the reply (backfill can take a while)
83+
await interaction.deferReply({ flags: MessageFlags.Ephemeral });
84+
85+
// Start the backfill operation
86+
const operationId = await backfillManager.startBackfill(interaction.user.id);
87+
logger.info(`[BACKFILL CMD] Operation ${operationId} started for channel ${targetChannel.name}`);
88+
89+
// Create progress tracker
90+
let progressMessage = '';
91+
const updateProgress = (message: string) => {
92+
progressMessage = message;
93+
logger.info(`[BACKFILL] ${message}`);
94+
};
95+
96+
// Run the backfill
97+
const stats = await backfillChannelHistory(targetChannel, updateProgress);
98+
99+
// Update operation status
100+
await backfillManager.updateBackfill(
101+
operationId,
102+
stats.codesFound,
103+
stats.codesRedeemed,
104+
stats.errors.length === 0 ? 'completed' : 'failed'
105+
);
106+
107+
// Create result embed
108+
const embed = new EmbedBuilder()
109+
.setColor(stats.errors.length === 0 ? 0x00aa00 : 0xffaa00)
110+
.setTitle('✅ Backfill Complete')
111+
.setDescription(
112+
[
113+
`**Codes Found:** ${stats.codesFound}`,
114+
`**Codes Redeemed:** ${stats.codesRedeemed}`,
115+
`**Pending Codes:** ${stats.pendingCodes}`,
116+
].join('\n')
117+
);
118+
119+
if (stats.errors.length > 0) {
120+
embed.addFields({
121+
name: '⚠️ Errors',
122+
value: stats.errors.slice(0, 5).join('\n'), // Show first 5 errors
123+
});
124+
}
125+
126+
embed.setFooter({
127+
text: `Operation ID: ${operationId}`,
128+
});
129+
130+
await interaction.editReply({ embeds: [embed] });
131+
132+
logger.info(
133+
`[BACKFILL CMD] Operation ${operationId} completed: found=${stats.codesFound}, redeemed=${stats.codesRedeemed}`
134+
);
135+
} catch (error) {
136+
logger.error('[BACKFILL CMD] Command error:', error);
137+
138+
try {
139+
const embed = new EmbedBuilder()
140+
.setColor(0xff0000)
141+
.setTitle('❌ Error')
142+
.setDescription(
143+
`An error occurred during backfill: ${error instanceof Error ? error.message : String(error)}`
144+
);
145+
146+
if (interaction.deferred) {
147+
await interaction.editReply({ embeds: [embed] });
148+
} else {
149+
await interaction.reply({ embeds: [embed], flags: MessageFlags.Ephemeral });
150+
}
151+
} catch (replyError) {
152+
logger.error('[BACKFILL CMD] Failed to send error reply:', replyError);
153+
}
154+
}
155+
}

0 commit comments

Comments
 (0)