-
Notifications
You must be signed in to change notification settings - Fork 116
docs: add comprehensive guides for agents, cron jobs, GraphORM, and background workers #3264
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| # Agents.md | ||
|
|
||
| This file provides guidance to coding agents when working with code in this repository. | ||
|
|
||
| ## Essential Commands | ||
|
|
||
| **Development:** | ||
| - `pnpm run dev` - Start API server with hot reload on port 5000 | ||
| - `pnpm run dev:background` - Start background processor | ||
| - `pnpm run dev:temporal-worker` - Start Temporal worker | ||
| - `pnpm run dev:temporal-server` - Start Temporal server for local development | ||
|
|
||
| **Database:** | ||
| - `pnpm run db:migrate:latest` - Apply latest migrations | ||
| - `pnpm run db:migrate:reset` - Drop schema and rerun migrations | ||
| - `pnpm run db:seed:import` - Import seed data for local development | ||
| - `pnpm run db:migrate:make` - Generate new migration based on entity changes | ||
| - `pnpm run db:migrate:create` - Create empty migration file | ||
|
|
||
| **Building & Testing:** | ||
| - `pnpm run build` - Compile TypeScript to build directory | ||
| - `pnpm run lint` - Run ESLint (max 0 warnings) | ||
| - `pnpm run test` - Run full test suite with database reset | ||
| - `pnpm run cli` - Run CLI commands (e.g., `pnpm run cli api`) | ||
|
|
||
| **Individual Test Execution:** | ||
| - Remember to run only individual tests when possible for faster feedback | ||
| - `NODE_ENV=test npx jest __tests__/specific-test.ts --testEnvironment=node --runInBand` | ||
| - Use `--testEnvironment=node --runInBand` flags for database-dependent tests | ||
|
|
||
| ## High-Level Architecture | ||
|
|
||
| **Core Framework Stack:** | ||
| - **Fastify** - Web framework with plugins for CORS, helmet, cookies, rate limiting | ||
| - **Mercurius** - GraphQL server with caching, upload support, and subscriptions | ||
| - **TypeORM** - Database ORM with entity-based modeling and migrations | ||
| - **PostgreSQL** - Primary database with master/slave replication setup. Favor read replica when you're ok with eventually consistent data. | ||
| - **Redis** - Caching and pub/sub via ioRedisPool | ||
| - **Temporal** - Workflow orchestration for background jobs | ||
| - **ClickHouse** - Analytics and metrics storage | ||
|
|
||
| **Application Entry Points:** | ||
| - `src/index.ts` - Main Fastify server setup with GraphQL, auth, and middleware | ||
| - `bin/cli.ts` - CLI dispatcher supporting api, background, temporal, cron modes | ||
| - `src/background.ts` - Pub/Sub message handlers and background processing | ||
| - `src/cron.ts` - Scheduled task execution | ||
|
|
||
| **GraphQL Schema Organization:** | ||
| - `src/graphql.ts` - Combines all schema modules with transformers and directives | ||
| - `src/schema/` - GraphQL resolvers organized by domain (posts, users, feeds, etc.) | ||
| - `src/directive/` - Custom GraphQL directives for auth, rate limiting, URL processing | ||
| - **Docs**: See `src/graphorm/AGENTS.md` for comprehensive guide on using GraphORM to solve N+1 queries. GraphORM is the default and preferred method for all GraphQL query responses. Use GraphORM instead of TypeORM repositories for GraphQL resolvers to prevent N+1 queries and enforce best practices. | ||
|
|
||
| **Data Layer:** | ||
| - `src/entity/` - TypeORM entities defining database schema | ||
| - `src/migration/` - Database migrations for schema evolution | ||
| - `src/data-source.ts` - Database connection with replication configuration | ||
|
|
||
| **Core Services:** | ||
| - `src/Context.ts` - Request context with user, permissions, and data loaders | ||
| - `src/auth.ts` - Authentication middleware and user context resolution | ||
| - `src/dataLoaderService.ts` - Efficient batch loading for related entities | ||
| - `src/workers/` - Use workers for async, non-critical operations (notifications, reputation, external syncs). Prefer `TypedWorker` for type safety. Architecture uses Google Pub/Sub + CDC (Debezium) for reactive processing. See `src/workers/AGENTS.md` for more. | ||
| - `src/integrations/` - External service integrations (Slack, SendGrid, etc.) | ||
| - `src/cron/` - Scheduled cron jobs for maintenance and periodic tasks. One file per cron, registered in `index.ts`, deployed via `.infra/crons.ts` Pulumi config. Each cron exports a `Cron` object with `name` and `handler(DataSource, logger, pubsub)`. Run locally with `pnpm run cli cron <name>`. See `src/cron/AGENTS.md` for more. | ||
|
|
||
| **Type Safety & Validation:** | ||
| - We favor type safety throughout the codebase. Use TypeScript interfaces and types for compile-time type checking. | ||
| - **Zod schemas** are preferred for runtime validation, especially for input validation, API boundaries, and data parsing. Zod provides both type inference and runtime validation, making it ideal for verifying user input, API payloads, and external data sources. | ||
| - When possible, prefer Zod schemas over manual validation as they provide type safety, better error messages, and can be inferred to TypeScript types. | ||
|
|
||
| **Business Domains:** | ||
|
idoshamun marked this conversation as resolved.
|
||
| - **Content**: Posts, comments, bookmarks, feeds, sources | ||
| - **Users**: Authentication, preferences, profiles, user experience | ||
| - **Organizations**: Squad management, member roles, campaigns | ||
| - **Notifications**: Push notifications, email digests, alerts | ||
| - **Monetization**: Paddle subscription management, premium features | ||
| - **Squads**: Squad management, member roles, campaigns | ||
|
|
||
| **Testing Strategy:** | ||
|
rebelchris marked this conversation as resolved.
|
||
| - Jest with supertest for integration testing | ||
| - Database reset before each test run via pretest hook | ||
| - Fixtures in `__tests__/fixture/` for test data | ||
| - Mercurius integration testing for GraphQL endpoints | ||
|
|
||
| **Infrastructure Concerns:** | ||
| - OpenTelemetry for distributed tracing and metrics | ||
| - GrowthBook for feature flags and A/B testing | ||
| - OneSignal for push notifications | ||
| - Temporal workflows for async job processing | ||
| - Rate limiting and caching at multiple layers | ||
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| AGENTS.md |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This comes with nuance that it imports everything retro, maybe just say prefer the create for agent mode? it's good at filling structured stuff anyway
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I usually find it safer to use make, it's battle tested and deterministic. It's up to the developer to make sure their environment is up to date
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just afraid it will prompt ALL changes in a file *(pretty sure this is what happens now right?)