Skip to content

Commit 3002cb2

Browse files
authored
feat: add materialized archive infrastructure (#3767)
## Summary - add materialized archive generation for best post monthly/yearly periods with GraphQL reads - tighten archive eligibility to public, feed-visible content from active public sources - add a standalone retroactive backfill bin that reuses the same period materialization path as the cron ## Verification - pnpm run lint - NODE_ENV=test npx jest __tests__/archive.ts --testEnvironment=node --runInBand - pnpm run build
1 parent d990dc8 commit 3002cb2

20 files changed

Lines changed: 1762 additions & 3 deletions

.infra/crons.ts

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -182,4 +182,12 @@ export const crons: Cron[] = [
182182
name: 'clean-expired-better-auth-sessions',
183183
schedule: '0 3 * * *',
184184
},
185+
{
186+
name: 'materialize-monthly-best-post-archives',
187+
schedule: '10 0 1 * *',
188+
},
189+
{
190+
name: 'materialize-yearly-best-post-archives',
191+
schedule: '15 0 1 1 *',
192+
},
185193
];

AGENTS.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,7 @@ The migration generator compares entities against the local database schema. Ens
115115
- `z.literal([...])` in Zod 4.x supports arrays and validates that the value matches one of the array elements
116116
- For enum-like validation of string literals, both `z.literal([...])` and `z.enum([...])` work in Zod 4.x
117117
- Always consult the [Zod 4.x documentation](https://zod.dev) for the latest API
118+
- For TypeScript string enums used in Zod schemas, derive the tuple from the shared helper in `src/common/schema/utils.ts` instead of duplicating arrays or hand-written type guards. Keep the enum as the single source of truth.
118119
- When possible, prefer Zod schemas over manual validation as they provide type safety, better error messages, and can be inferred to TypeScript types.
119120
- **Connect RPC handlers must return typed proto message classes** from `@dailydotdev/schema`, not plain objects. Use `new ResponseType({...})` instead of returning `{...}` directly. **This applies to mock/`isMockEnabled` returns too** — when mocking RPC transport, always use actual proto message class instances, not raw JSON objects.
120121
- **Never create wrapper types around `@dailydotdev/schema` classes** (e.g., `UserBriefingRequest & { extraField }`) — if a field exists in the proto, use it directly. If it doesn't exist yet, update the schema package first.
@@ -576,6 +577,7 @@ The migration generator compares entities against the local database schema. Ens
576577
```
577578

578579
- **For cron jobs and batch operations**: Keep read-only queries outside the transaction, then wrap all writes in a single transaction.
580+
- For materialization/backfill jobs, make the cron rerunnable and idempotent. Prefer a deterministic unique key plus an atomic per-scope write step so retries can safely continue after partial progress without leaving half-written rows.
579581

580582
**Using queryReadReplica Helper:**
581583

@@ -598,6 +600,11 @@ The migration generator compares entities against the local database schema. Ens
598600
- For integration tests that depend on materialized views, assume schema setup is handled by migrations (`db:migrate:latest` / test reset flow).
599601
- In tests, refresh the materialized view before assertions; do not recreate the materialized view definition inside test files.
600602

603+
**Immutable Materialized Tables:**
604+
605+
- For immutable or period-closed materialized data, prefer the smallest persisted schema that supports lookup and joins. Avoid `updatedAt`, `generatedAt`, counters, or snapshot columns unless there is a concrete read-path need for them.
606+
- For archive/materialization tables keyed by business dimensions, keep a surrogate `id` for relations but also enforce the real unique business key in the database.
607+
601608
**State Checking Patterns:**
602609

603610
- **Prefer negative checks over listing states** when checking for "non-draft" or similar conditions.

0 commit comments

Comments
 (0)