Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 22 additions & 8 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,15 @@

This file provides guidance to coding agents when working with code in this repository.

## Prerequisites

- **Node.js**: 22.16.0 (managed via Volta)
- **Package Manager**: pnpm 9.14.4

## Essential Commands

**Development:**
- `pnpm run dev` - Start API server with hot reload on port 5000
- `pnpm run dev` - Start API server with hot reload on port 3000
- `pnpm run dev:background` - Start background processor
- `pnpm run dev:temporal-worker` - Start Temporal worker
- `pnpm run dev:temporal-server` - Start Temporal server for local development
Expand Down Expand Up @@ -35,15 +40,17 @@ This file provides guidance to coding agents when working with code in this repo
- **Mercurius** - GraphQL server with caching, upload support, and subscriptions
- **TypeORM** - Database ORM with entity-based modeling and migrations
- **PostgreSQL** - Primary database with master/slave replication setup. Favor read replica when you're ok with eventually consistent data.
- **Redis** - Caching and pub/sub via ioRedisPool
- **Redis** - Caching and pub/sub via `@dailydotdev/ts-ioredis-pool`
- **Temporal** - Workflow orchestration for background jobs
- **ClickHouse** - Analytics and metrics storage

**Application Entry Points:**
- `src/index.ts` - Main Fastify server setup with GraphQL, auth, and middleware
- `bin/cli.ts` - CLI dispatcher supporting api, background, temporal, cron modes
- `bin/cli.ts` - CLI dispatcher supporting api, background, temporal, cron, personalized-digest modes
- `src/background.ts` - Pub/Sub message handlers and background processing
- `src/cron.ts` - Scheduled task execution
- `src/temporal/` - Temporal workflow definitions and workers
- `src/commands/` - Standalone command implementations (e.g., personalized digest)

**GraphQL Schema Organization:**
- `src/graphql.ts` - Combines all schema modules with transformers and directives
Expand Down Expand Up @@ -71,11 +78,12 @@ This file provides guidance to coding agents when working with code in this repo

**Business Domains:**
- **Content**: Posts, comments, bookmarks, feeds, sources
- **Users**: Authentication, preferences, profiles, user experience
- **Organizations**: Squad management, member roles, campaigns
- **Users**: Authentication, preferences, profiles, user experience, streaks
- **Squads**: Squad management, member roles, public requests
- **Organizations**: Organization management, campaigns
- **Notifications**: Push notifications, email digests, alerts
- **Monetization**: Paddle subscription management, premium features
- **Squads**: Squad management, member roles, campaigns
- **Monetization**: Paddle subscription management, premium features, cores/transactions
- **Opportunities**: Job matching, recruiter features, candidate profiles

**Testing Strategy:**
- Jest with supertest for integration testing
Expand All @@ -88,4 +96,10 @@ This file provides guidance to coding agents when working with code in this repo
- GrowthBook for feature flags and A/B testing
- OneSignal for push notifications
- Temporal workflows for async job processing
- Rate limiting and caching at multiple layers
- Rate limiting and caching at multiple layers

**Infrastructure as Code:**
- `.infra/` - Pulumi configuration for deployment
- `.infra/crons.ts` - Cron job schedules and resource limits
- `.infra/common.ts` - Worker subscription definitions
- `.infra/index.ts` - Main Pulumi deployment configuration
55 changes: 32 additions & 23 deletions src/cron/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ Create a new file in `src/cron/` directory following this pattern:
import { Cron } from './cron';
import { YourEntity } from '../entity';

const cron: Cron = {
export const yourCronName: Cron = {
name: 'your-cron-name', // Must match name in .infra/crons.ts
handler: async (con, logger, pubsub) => {
// Your cron logic here
Expand All @@ -75,20 +75,18 @@ const cron: Cron = {
logger.info({ count: results.length }, 'Cron job completed');
},
};

export default cron;
```

### Step 2: Register in Index

Add your cron to `src/cron/index.ts`:

```typescript
import yourCron from './yourCron';
import { yourCronName } from './yourCronName';

export const crons: Cron[] = [
// ... existing crons
yourCron,
yourCronName,
];
```

Expand Down Expand Up @@ -202,21 +200,32 @@ handler: async (con, logger, pubsub) => {

### 6. Testing

Create tests in `__tests__/cron/` directory:
Create tests in `__tests__/cron/` directory. Tests use a real database connection (reset before each test run):

```typescript
import cron from '../../src/cron/yourCron';
import { yourCronName } from '../../src/cron/yourCronName';
import { expectSuccessfulCron, saveFixtures } from '../helpers';
import { YourEntity } from '../../src/entity';
import { DataSource } from 'typeorm';
import createOrGetConnection from '../../src/db';

describe('yourCron', () => {
it('should execute successfully', async () => {
const mockCon = createMockConnection();
const mockLogger = createMockLogger();
const mockPubsub = createMockPubsub();

await cron.handler(mockCon, mockLogger, mockPubsub);

// Assertions
});
let con: DataSource;

beforeAll(async () => {
con = await createOrGetConnection();
});

beforeEach(async () => {
// Set up test fixtures
await saveFixtures(con, YourEntity, yourFixtures);
});

it('should execute successfully', async () => {
await expectSuccessfulCron(yourCronName);

// Verify database state after cron execution
const results = await con.getRepository(YourEntity).find();
expect(results).toHaveLength(expectedLength);
});
```

Expand All @@ -225,7 +234,7 @@ describe('yourCron', () => {
### Incremental Processing with Checkpoints

```typescript
const cron: Cron = {
export const incrementalUpdate: Cron = {
name: 'incremental-update',
handler: async (con) => {
const checkpointKey = 'last_incremental_update';
Expand Down Expand Up @@ -290,13 +299,13 @@ handler: async (con, logger) => {

### Kubernetes CronJobs

Crons are deployed as Kubernetes CronJobs via Pulumi:
Crons are deployed as Kubernetes CronJobs via Pulumi (see `.infra/index.ts` lines 577-591):

- **Command**: `['dumb-init', 'node', 'bin/cli', 'cron', '<cron-name>']`
- **Spot Instances**: Enabled by default (70% weight) for cost optimization
- **Default Limits**: Memory and CPU limits can be overridden per cron
- **Default Deadline**: 300 seconds (5 minutes), configurable per cron
- **Adhoc Environments**: Crons are disabled in adhoc environments (see `.infra/index.ts`)
- **Spot Instances**: Enabled by default for all crons
- **Default Limits**: Uses background worker limits (`512Mi` memory) unless overridden per cron
- **Default Deadline**: 300 seconds (5 minutes), configurable per cron via `activeDeadlineSeconds`
- **Adhoc Environments**: Crons are disabled in adhoc environments

### Resource Configuration

Expand Down
69 changes: 61 additions & 8 deletions src/graphorm/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,9 @@ export const resolvers = {
};
```

### Query with Custom Filtering
### Query with Custom Filtering and Read Replica

The optional fourth parameter enables read replica routing for eventually consistent reads:

```typescript
export const resolvers = {
Expand All @@ -239,7 +241,7 @@ export const resolvers = {
.limit(1);
return builder;
},
true // Use read replica
true // for better performance for read queries are required, take into account potential replication lag if doing reads right after writes
);
}
}
Expand All @@ -265,6 +267,42 @@ export const resolvers = {
};
```

### Query by Hierarchy

Use `queryByHierarchy` when you need to query a nested field from the resolve tree:

```typescript
export const resolvers = {
Query: {
searchPosts: async (_, args, ctx: Context, info) => {
return graphorm.queryByHierarchy<GQLPost>(
ctx,
info,
['posts', 'edges', 'node'], // Path to the nested field
(builder) => {
builder.queryBuilder.where(`${builder.alias}.visible = true`);
return builder;
},
true // readReplica
);
}
}
};
```

### Query Paginated Integration

Use `queryPaginatedIntegration` for non-database data (e.g., external APIs) while still returning Relay-style pagination:

```typescript
const results = await graphorm.queryPaginatedIntegration<ExternalItem>(
(nodeSize) => false, // hasPreviousPage
(nodeSize) => nodeSize >= limit, // hasNextPage
(node, index) => base64(`cursor:${index}`), // nodeToCursor
async () => fetchFromExternalAPI(args), // fetchData callback
);
```

## Configuration Patterns

### Required Columns
Expand Down Expand Up @@ -354,11 +392,12 @@ fields: {

## Best Practices

### 1. Always Use `beforeQuery` for Filtering
### 1. Always Use the Query Builder Callback for Filtering

Don't fetch all data and filter in JavaScript. Filter at the database level:
Don't fetch all data and filter in JavaScript. Filter at the database level using the builder callback parameter:

```typescript
// The callback receives { queryBuilder, alias } and must return the modified builder
(builder) => {
builder.queryBuilder
.andWhere(`${builder.alias}."userId" = :userId`, { userId: ctx.userId })
Expand Down Expand Up @@ -395,10 +434,13 @@ Always paginate lists to avoid fetching too much data:
graphorm.queryPaginated(
ctx,
info,
hasPreviousPage,
hasNextPage,
nodeToCursor,
beforeQuery
(nodeSize) => hasPreviousPage(nodeSize),
(nodeSize) => hasNextPage(nodeSize),
(node, index) => nodeToCursor(node, index),
(builder) => {
builder.queryBuilder.limit(limit).offset(offset);
return builder;
}
);
```

Expand Down Expand Up @@ -450,6 +492,17 @@ requiredColumns: [
4. **Required Columns**: Only add truly required columns to avoid unnecessary data fetching
5. **Transform Functions**: Keep transforms lightweight - avoid database calls in transforms

## Available Methods

| Method | Description | Returns |
|--------|-------------|---------|
| `query<T>()` | Query multiple results | `Promise<T[]>` |
| `queryOne<T>()` | Query single result or null | `Promise<T \| null>` |
| `queryOneOrFail<T>()` | Query single result or throw | `Promise<T>` |
| `queryPaginated<T>()` | Query with Relay pagination | `Promise<Connection<T>>` |
| `queryByHierarchy<T>()` | Query nested field from resolve tree | `Promise<T[]>` |
| `queryPaginatedIntegration<T>()` | Paginate non-DB data | `Promise<Connection<T>>` |

## Related Files

- **Core Implementation**: `src/graphorm/graphorm.ts`
Expand Down
Loading
Loading