Skip to content

Commit 0384013

Browse files
mcowgerclaude
andauthored
Update Quota Tracking (#301)
Replaces the old QuotaWindow[]-based system with a flat Meter[] model where each meter is self-describing with kind, unit, period fields, and status — eliminating per-checker UI switches, checker-level categories, compound enum window types, and 21 near-duplicate display components. Backend changes: - New Meter/MeterCheckResult types in src/types/meter.ts - Self-registering checker-registry.ts (defineChecker + loadAllCheckers) - New meter_snapshots table schema (SQLite + PostgreSQL) - All 22 checkers rewritten to export default defineChecker({...}) - apertis and apertis-coding-plan merged into single apertis checker - quota-scheduler.ts rewired to use registry + persist to meter_snapshots - /v0/management/quotas routes return MeterCheckResult shape - apertis-coding-plan removed from config schema Frontend changes: - New QuotaCheckerInfo type with meters: Meter[] replacing latest: snapshot[] - New generic components: BalanceMeterRow, AllowanceMeterRow, MeterValue - checker-presentation.ts for display names - Quotas.tsx, CombinedBalancesCard, CompactQuotasCard, CompactBalancesCard rewritten to use meters array directly - Sidebar.tsx updated to derive balance/allowance from meter kinds - All 21 XxxQuotaDisplay.tsx components deleted - QuotaHistoryModal, BalanceHistoryModal deleted - api.ts updated to use new response shape --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 0b30d18 commit 0384013

96 files changed

Lines changed: 4531 additions & 8865 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -211,6 +211,69 @@ On first startup with `ENCRYPTION_KEY` set, existing plaintext values are automa
211211

212212
---
213213

214+
## Admin CLI Utilities
215+
216+
Plexus ships several one-shot CLI subcommands for database maintenance tasks. Pass the subcommand name as the first argument to the binary (or `bun run src/index.ts`).
217+
218+
### Rotate Encryption Key (`rekey`)
219+
220+
Decrypts all sensitive fields with the current key and re-encrypts them with a new one. Run this before rotating `ENCRYPTION_KEY` in your environment.
221+
222+
```bash
223+
# Docker
224+
docker run --rm \
225+
-e DATABASE_URL=sqlite:///app/data/plexus.db \
226+
-e ENCRYPTION_KEY="<current-key>" \
227+
-e NEW_ENCRYPTION_KEY="<new-key>" \
228+
-v plexus-data:/app/data \
229+
ghcr.io/mcowger/plexus:latest rekey
230+
231+
# Binary
232+
ENCRYPTION_KEY="<current-key>" NEW_ENCRYPTION_KEY="<new-key>" \
233+
DATABASE_URL=sqlite://./data/plexus.db ./plexus rekey
234+
```
235+
236+
After a successful run, update `ENCRYPTION_KEY` to the new value before restarting the server.
237+
238+
→ See [Configuration: Encryption](docs/CONFIGURATION.md#encryption-at-rest-optional)
239+
240+
### Migrate Quota Snapshots (`migrate-quota-snapshots`)
241+
242+
One-time ETL that copies historical data from the legacy `quota_snapshots` table into the new `meter_snapshots` table introduced in the quota-tracking overhaul. Run this once after upgrading to a version that includes the new quota system.
243+
244+
```bash
245+
# Docker
246+
docker run --rm \
247+
-e DATABASE_URL=sqlite:///app/data/plexus.db \
248+
-v plexus-data:/app/data \
249+
ghcr.io/mcowger/plexus:latest migrate-quota-snapshots
250+
251+
# Binary
252+
DATABASE_URL=sqlite://./data/plexus.db ./plexus migrate-quota-snapshots
253+
254+
# Development
255+
DATABASE_URL=sqlite://./data/plexus.db bun run src/index.ts migrate-quota-snapshots
256+
```
257+
258+
`DATABASE_URL` must be set explicitly — there is no default. The command is **idempotent**: rows that already exist in `meter_snapshots` are skipped, so it is safe to run more than once. If `quota_snapshots` does not exist or is empty the command exits cleanly with no changes.
259+
260+
**Field mapping summary:**
261+
262+
| `quota_snapshots` | `meter_snapshots` | Notes |
263+
|---|---|---|
264+
| `provider` | `provider` | direct |
265+
| `checker_id` | `checker_id` | direct |
266+
| `group_id` | `group` | renamed |
267+
| `window_type` | `meter_key` | used as-is |
268+
| `window_type` | `kind` / `period_*` | `daily`→allowance/day, `monthly`→allowance/month, `balance`→balance, etc. |
269+
| `description` | `label` | falls back to `window_type` if null |
270+
| `unit` | `unit` | defaults to `''` if null |
271+
| `status` | `status` | defaults to `'ok'` if null |
272+
| `utilization_percent` | `utilization_percent` + `utilization_state` | null→`unknown`, number→`reported` |
273+
| *(not present)* | `checker_type` | set to `'unknown'` |
274+
275+
---
276+
214277
## License
215278

216279
MIT License — see [LICENSE](LICENSE) file

packages/backend/drizzle/schema/index.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Re-export from sqlite by default for backward compatibility
2+
export * from './sqlite/meter-snapshots';
23
export * from './sqlite/request-usage';
34
export * from './sqlite/provider-cooldowns';
45
export * from './sqlite/debug-logs';
@@ -21,6 +22,7 @@ export { debugLogs as pgDebugLogs } from './postgres/debug-logs';
2122
export { inferenceErrors as pgInferenceErrors } from './postgres/inference-errors';
2223
export { providerPerformance as pgProviderPerformance } from './postgres/provider-performance';
2324
export { quotaSnapshots as pgQuotaSnapshots } from './postgres/quota-snapshots';
25+
export { meterSnapshots as pgMeterSnapshots } from './postgres/meter-snapshots';
2426
export {
2527
responses as pgResponses,
2628
conversations as pgConversations,

packages/backend/drizzle/schema/postgres/index.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
export * from './meter-snapshots';
12
export * from './request-usage';
23
export * from './provider-cooldowns';
34
export * from './debug-logs';
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
import { pgTable, serial, text, real, integer, bigint, index } from 'drizzle-orm/pg-core';
2+
3+
export const meterSnapshots = pgTable(
4+
'meter_snapshots',
5+
{
6+
id: serial('id').primaryKey(),
7+
checkerId: text('checker_id').notNull(),
8+
checkerType: text('checker_type').notNull(),
9+
provider: text('provider').notNull(),
10+
meterKey: text('meter_key').notNull(),
11+
kind: text('kind').notNull(), // 'balance' | 'allowance'
12+
unit: text('unit').notNull(),
13+
label: text('label').notNull(),
14+
group: text('group'),
15+
scope: text('scope'),
16+
limit: real('limit'),
17+
used: real('used'),
18+
remaining: real('remaining'),
19+
utilizationState: text('utilization_state').notNull(), // 'reported' | 'unknown' | 'not_applicable'
20+
utilizationPercent: real('utilization_percent'),
21+
status: text('status').notNull(),
22+
periodValue: integer('period_value'),
23+
periodUnit: text('period_unit'),
24+
periodCycle: text('period_cycle'),
25+
resetsAt: bigint('resets_at', { mode: 'number' }),
26+
success: integer('success').notNull().default(1),
27+
errorMessage: text('error_message'),
28+
checkedAt: bigint('checked_at', { mode: 'number' }).notNull(),
29+
createdAt: bigint('created_at', { mode: 'number' }).notNull(),
30+
},
31+
(table) => ({
32+
checkerMeterCheckedIdx: index('idx_meter_checker_meter_checked').on(
33+
table.checkerId,
34+
table.meterKey,
35+
table.checkedAt
36+
),
37+
providerCheckedIdx: index('idx_meter_provider_checked').on(table.provider, table.checkedAt),
38+
checkedAtIdx: index('idx_meter_checked_at').on(table.checkedAt),
39+
})
40+
);

packages/backend/drizzle/schema/sqlite/index.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
export * from './meter-snapshots';
12
export * from './request-usage';
23
export * from './provider-cooldowns';
34
export * from './debug-logs';
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
import { sqliteTable, integer, text, real, index } from 'drizzle-orm/sqlite-core';
2+
3+
export const meterSnapshots = sqliteTable(
4+
'meter_snapshots',
5+
{
6+
id: integer('id').primaryKey({ autoIncrement: true }),
7+
checkerId: text('checker_id').notNull(),
8+
checkerType: text('checker_type').notNull(),
9+
provider: text('provider').notNull(),
10+
meterKey: text('meter_key').notNull(),
11+
kind: text('kind').notNull(), // 'balance' | 'allowance'
12+
unit: text('unit').notNull(),
13+
label: text('label').notNull(),
14+
group: text('group'),
15+
scope: text('scope'),
16+
limit: real('limit'),
17+
used: real('used'),
18+
remaining: real('remaining'),
19+
utilizationState: text('utilization_state').notNull(), // 'reported' | 'unknown' | 'not_applicable'
20+
utilizationPercent: real('utilization_percent'),
21+
status: text('status').notNull(),
22+
periodValue: integer('period_value'),
23+
periodUnit: text('period_unit'),
24+
periodCycle: text('period_cycle'),
25+
resetsAt: integer('resets_at', { mode: 'timestamp_ms' }),
26+
success: integer('success', { mode: 'boolean' }).notNull().default(true),
27+
errorMessage: text('error_message'),
28+
checkedAt: integer('checked_at', { mode: 'timestamp_ms' }).notNull(),
29+
createdAt: integer('created_at', { mode: 'timestamp_ms' }).notNull(),
30+
},
31+
(table) => ({
32+
checkerMeterCheckedIdx: index('idx_meter_checker_meter_checked').on(
33+
table.checkerId,
34+
table.meterKey,
35+
table.checkedAt
36+
),
37+
providerCheckedIdx: index('idx_meter_provider_checked').on(table.provider, table.checkedAt),
38+
checkedAtIdx: index('idx_meter_checked_at').on(table.checkedAt),
39+
})
40+
);
Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
#!/usr/bin/env bun
2+
/**
3+
* One-time ETL: migrate data from the legacy `quota_snapshots` table to the
4+
* new `meter_snapshots` table introduced in the quota-tracking overhaul.
5+
*
6+
* Usage:
7+
* bun run src/cli/migrate-quota-snapshots.ts
8+
*
9+
* The DATABASE_URL env var must be set explicitly — there is no default fallback.
10+
*
11+
* This script is idempotent — rows already present in `meter_snapshots` for
12+
* a given (checkerId, meterKey, checkedAt) triplet are skipped, so it is safe
13+
* to run more than once.
14+
*
15+
* Field mapping from quota_snapshots → meter_snapshots:
16+
* provider → provider
17+
* checker_id → checker_id (checkerId)
18+
* group_id → group (renamed)
19+
* window_type → kind, period* (derived – see mapWindowType)
20+
* window_type → meter_key (used as a stable key)
21+
* description → label (falls back to window_type)
22+
* checked_at → checked_at
23+
* limit → limit
24+
* used → used
25+
* remaining → remaining
26+
* utilization_% → utilization_percent + utilization_state
27+
* unit → unit (defaults to '' if null)
28+
* resets_at → resets_at
29+
* status → status (defaults to 'ok' if null)
30+
* success → success
31+
* error_message → error_message
32+
* created_at → created_at
33+
*
34+
* checker_type → 'unknown' (not stored in old table)
35+
*/
36+
37+
import { initializeDatabase } from '../db/client';
38+
import { runMigrations } from '../db/migrate';
39+
import { logger } from '../utils/logger';
40+
import { migrateLegacySnapshots } from '../services/quota/legacy-snapshot-migrator';
41+
42+
async function main() {
43+
if (!process.env.DATABASE_URL) {
44+
logger.error('DATABASE_URL must be set');
45+
process.exit(1);
46+
}
47+
48+
initializeDatabase();
49+
await runMigrations();
50+
51+
const { inserted, skipped, totalSource } = await migrateLegacySnapshots();
52+
53+
if (totalSource === 0) {
54+
logger.info('Nothing to migrate.');
55+
} else {
56+
logger.info(`Migration complete. Inserted: ${inserted}, Skipped: ${skipped}`);
57+
}
58+
}
59+
60+
export { main as migrateQuotaSnapshotsMain };
61+
62+
if (import.meta.main) {
63+
main().catch((err) => {
64+
logger.error('Quota snapshot migration failed:', err);
65+
process.exit(1);
66+
});
67+
}

packages/backend/src/config.ts

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -301,13 +301,6 @@ const ProviderQuotaCheckerSchema = z.discriminatedUnion('type', [
301301
id: z.string().trim().min(1).optional(),
302302
options: ApertisQuotaCheckerOptionsSchema.optional().default({}),
303303
}),
304-
z.object({
305-
type: z.literal('apertis-coding-plan'),
306-
enabled: z.boolean().default(true),
307-
intervalMinutes: z.number().min(1).default(30),
308-
id: z.string().trim().min(1).optional(),
309-
options: ApertisQuotaCheckerOptionsSchema.optional().default({}),
310-
}),
311304
z.object({
312305
type: z.literal('minimax-coding'),
313306
enabled: z.boolean().default(true),
@@ -905,7 +898,6 @@ export const VALID_QUOTA_CHECKER_TYPES = [
905898
'copilot',
906899
'wisdomgate',
907900
'apertis',
908-
'apertis-coding-plan',
909901
'poe',
910902
'gemini-cli',
911903
'antigravity',

packages/backend/src/db/types.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,8 @@ export type ProviderCooldown = InferSelectModel<typeof schema.providerCooldowns>
66
export type DebugLog = InferSelectModel<typeof schema.debugLogs>;
77
export type InferenceError = InferSelectModel<typeof schema.inferenceErrors>;
88
export type ProviderPerformance = InferSelectModel<typeof schema.providerPerformance>;
9-
export type QuotaSnapshot = InferSelectModel<typeof schema.quotaSnapshots>;
9+
// QuotaSnapshot / NewQuotaSnapshot removed — the old quota_snapshots table is
10+
// superseded by meter_snapshots. See src/types/meter.ts for the new types.
1011
export type McpRequestUsage = InferSelectModel<typeof schema.mcpRequestUsage>;
1112
export type McpDebugLog = InferSelectModel<typeof schema.mcpDebugLogs>;
1213

@@ -15,7 +16,6 @@ export type NewProviderCooldown = InferInsertModel<typeof schema.providerCooldow
1516
export type NewDebugLog = InferInsertModel<typeof schema.debugLogs>;
1617
export type NewInferenceError = InferInsertModel<typeof schema.inferenceErrors>;
1718
export type NewProviderPerformance = InferInsertModel<typeof schema.providerPerformance>;
18-
export type NewQuotaSnapshot = InferInsertModel<typeof schema.quotaSnapshots>;
1919
export type NewMcpRequestUsage = InferInsertModel<typeof schema.mcpRequestUsage>;
2020
export type NewMcpDebugLog = InferInsertModel<typeof schema.mcpDebugLogs>;
2121

packages/backend/src/index.ts

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,17 @@ if (subcommand === 'rekey') {
1313
await new Promise(() => {}); // Block forever; process.exit above will terminate
1414
}
1515

16+
if (subcommand === 'migrate-quota-snapshots') {
17+
const { migrateQuotaSnapshotsMain } = await import('./cli/migrate-quota-snapshots');
18+
migrateQuotaSnapshotsMain()
19+
.then(() => process.exit(0))
20+
.catch((err) => {
21+
console.error('Quota snapshot migration failed:', err);
22+
process.exit(1);
23+
});
24+
await new Promise(() => {});
25+
}
26+
1627
import Fastify, { FastifyReply, FastifyRequest } from 'fastify';
1728
import cors from '@fastify/cors';
1829
import multipart from '@fastify/multipart';

0 commit comments

Comments
 (0)