|
| 1 | +--- |
| 2 | +name: adding-functions |
| 3 | +description: Step-by-step guide for adding a new serverless function to the constructive-functions project |
| 4 | +--- |
| 5 | + |
| 6 | +# Adding a New Function |
| 7 | + |
| 8 | +## Prerequisites |
| 9 | + |
| 10 | +- Node.js 22+, pnpm 10+ |
| 11 | +- Understanding of the `FunctionHandler` type from `@constructive-io/fn-runtime` |
| 12 | + |
| 13 | +**Reference implementations:** See `functions/simple-email/` (env vars, external packages, dry-run mode) and `functions/send-email-link/` (GraphQL queries, context usage) as working examples. |
| 14 | + |
| 15 | +## Step 1: Create handler.json |
| 16 | + |
| 17 | +Create `functions/<name>/handler.json`: |
| 18 | + |
| 19 | +```json |
| 20 | +{ |
| 21 | + "name": "<name>", |
| 22 | + "version": "1.0.0", |
| 23 | + "type": "node-graphql", |
| 24 | + "port": <next-available-port>, |
| 25 | + "description": "What this function does", |
| 26 | + "dependencies": { |
| 27 | + "some-package": "^1.0.0" |
| 28 | + } |
| 29 | +} |
| 30 | +``` |
| 31 | + |
| 32 | +### handler.json fields |
| 33 | + |
| 34 | +| Field | Required | Description | |
| 35 | +|-------|----------|-------------| |
| 36 | +| `name` | Yes | Function identifier (used in job queue, k8s service names, Docker images) | |
| 37 | +| `version` | Yes | Semver version | |
| 38 | +| `type` | No | Template type, defaults to `node-graphql` | |
| 39 | +| `port` | No | Local dev port (auto-assigned from 8081+ if omitted) | |
| 40 | +| `description` | No | Human-readable description | |
| 41 | +| `dependencies` | No | NPM dependencies merged into the generated package.json | |
| 42 | + |
| 43 | +**Naming convention:** The `name` field is the canonical identifier — it's used for job queue task names, k8s service/deployment names, and the generated package name (`@constructive-io/<name>-fn`). The directory name under `functions/` is just for local organization. They don't have to match (e.g., `functions/example/` has `"name": "knative-job-example"`), but keeping them consistent avoids confusion. |
| 44 | + |
| 45 | +**Port convention:** Check existing `functions/*/handler.json` files for used ports. Pick the next available (8081, 8082, 8083, ...). Port 8080 is reserved for job-service. |
| 46 | + |
| 47 | +## Step 2: Create handler.ts |
| 48 | + |
| 49 | +Create `functions/<name>/handler.ts`: |
| 50 | + |
| 51 | +```typescript |
| 52 | +import type { FunctionHandler } from '@constructive-io/fn-runtime'; |
| 53 | + |
| 54 | +interface MyPayload { |
| 55 | + // Define your expected job payload |
| 56 | +} |
| 57 | + |
| 58 | +const handler: FunctionHandler<MyPayload> = async (params, context) => { |
| 59 | + // context provides: { client, meta, job, log, env } |
| 60 | + // client — GraphQL client for the database's API |
| 61 | + // meta — GraphQL client for metadata API |
| 62 | + // job — { jobId, workerId, databaseId } |
| 63 | + // log — structured logger (info, error, warn, debug) |
| 64 | + // env — process.env |
| 65 | + |
| 66 | + // Your implementation here |
| 67 | + |
| 68 | + return { complete: true }; |
| 69 | +}; |
| 70 | + |
| 71 | +export default handler; |
| 72 | +``` |
| 73 | + |
| 74 | +**Key patterns:** |
| 75 | +- Return `{ complete: true }` on success — the job service marks the job done |
| 76 | +- Throw an error on failure — the job service retries with backoff |
| 77 | +- Return an error object like `{ missing: 'field' }` for validation failures that should not retry |
| 78 | + |
| 79 | +If your function imports modules that need TypeScript type stubs, add a `types.d.ts` in the function directory: |
| 80 | + |
| 81 | +```typescript |
| 82 | +declare module '@some-untyped-package'; |
| 83 | +``` |
| 84 | + |
| 85 | +## Step 3: Register with the job service |
| 86 | + |
| 87 | +Update `job/service/src/types.ts` — add the function name to the `FunctionName` union: |
| 88 | + |
| 89 | +```typescript |
| 90 | +export type FunctionName = 'simple-email' | 'send-email-link' | '<name>'; |
| 91 | +``` |
| 92 | + |
| 93 | +Update `job/service/src/index.ts` — add an entry to `functionRegistry`: |
| 94 | + |
| 95 | +```typescript |
| 96 | +'<name>': { |
| 97 | + moduleName: '@constructive-io/<name>-fn', |
| 98 | + defaultPort: <port> |
| 99 | +}, |
| 100 | +``` |
| 101 | + |
| 102 | +The `moduleName` is the generated workspace package name (`@constructive-io/<name>-fn`). The `defaultPort` must match the port in `handler.json`. |
| 103 | + |
| 104 | +## Step 4: Run generate |
| 105 | + |
| 106 | +```bash |
| 107 | +pnpm generate |
| 108 | +``` |
| 109 | + |
| 110 | +This produces everything in `generated/<name>/`: |
| 111 | +- `package.json` — workspace package with merged dependencies |
| 112 | +- `index.ts` — Express wrapper around your handler |
| 113 | +- `tsconfig.json` + `tsconfig.esm.json` — TypeScript config |
| 114 | +- `Dockerfile` — multi-stage production build |
| 115 | +- `k8s/local-deployment.yaml` — K8s Deployment + Service for local dev |
| 116 | +- `k8s/knative-service.yaml` — Knative Service for production |
| 117 | +- `k8s/skaffold-overlay/` — per-function kustomize overlay for Skaffold |
| 118 | +- `README.md` |
| 119 | +- `handler.ts` — symlink to your source |
| 120 | + |
| 121 | +It also updates: |
| 122 | +- `skaffold.yaml` — adds a per-function profile and updates aggregate profiles |
| 123 | +- `k8s/overlays/local-simple/job-service.yaml` — adds function to JOBS_SUPPORTED and gateway map |
| 124 | +- `generated/functions-manifest.json` — function registry used by dev.ts |
| 125 | + |
| 126 | +## Step 5: Install and build |
| 127 | + |
| 128 | +```bash |
| 129 | +pnpm install # picks up the new workspace package |
| 130 | +pnpm build # builds all packages including the new function |
| 131 | +``` |
| 132 | + |
| 133 | +## Step 6: Add unit tests |
| 134 | + |
| 135 | +Create `functions/<name>/__tests__/handler.test.ts`: |
| 136 | + |
| 137 | +```typescript |
| 138 | +import { createMockContext } from '../../../tests/helpers/mock-context'; |
| 139 | + |
| 140 | +const loadHandler = () => { |
| 141 | + const mod = require('../handler'); |
| 142 | + return mod.default ?? mod; |
| 143 | +}; |
| 144 | + |
| 145 | +describe('<name> handler', () => { |
| 146 | + beforeEach(() => { |
| 147 | + jest.resetModules(); |
| 148 | + }); |
| 149 | + |
| 150 | + it('should process valid payload', async () => { |
| 151 | + const handler = loadHandler(); |
| 152 | + const result = await handler({ /* test payload */ }, createMockContext()); |
| 153 | + expect(result).toEqual({ complete: true }); |
| 154 | + }); |
| 155 | + |
| 156 | + it('should reject invalid payload', async () => { |
| 157 | + const handler = loadHandler(); |
| 158 | + await expect( |
| 159 | + handler({}, createMockContext()) |
| 160 | + ).rejects.toThrow(); |
| 161 | + }); |
| 162 | +}); |
| 163 | +``` |
| 164 | + |
| 165 | +**Why `require()` + `resetModules()`:** Handlers often read env vars at module scope (e.g., `parseEnvBoolean(process.env.SOME_FLAG)`). Using `require()` with `jest.resetModules()` ensures each test gets a fresh module evaluation, so env var changes in `beforeEach` take effect. |
| 166 | + |
| 167 | +Use `tests/helpers/mock-context.ts` to create test contexts. If your function uses external packages, add mocks in `tests/__mocks__/` and register them in `jest.config.ts` under `moduleNameMapper`. |
| 168 | + |
| 169 | +Run: `pnpm test:unit` |
| 170 | + |
| 171 | +## Step 7: Add e2e test |
| 172 | + |
| 173 | +Create `tests/e2e/__tests__/<name>.e2e.test.ts`: |
| 174 | + |
| 175 | +```typescript |
| 176 | +import { |
| 177 | + getTestConnections, |
| 178 | + closeConnections, |
| 179 | + getDatabaseId, |
| 180 | + TestClient, |
| 181 | +} from '../utils/db'; |
| 182 | +import { addJob, waitForJobComplete, deleteTestJobs } from '../utils/jobs'; |
| 183 | + |
| 184 | +const TEST_PREFIX = 'k8s-e2e-<name>'; |
| 185 | + |
| 186 | +describe('E2E: <name>', () => { |
| 187 | + let pg: TestClient; |
| 188 | + let databaseId: string; |
| 189 | + |
| 190 | + beforeAll(async () => { |
| 191 | + const connections = await getTestConnections(); |
| 192 | + pg = connections.pg; |
| 193 | + databaseId = await getDatabaseId(pg); |
| 194 | + }); |
| 195 | + |
| 196 | + afterAll(async () => { |
| 197 | + if (pg) await deleteTestJobs(pg, TEST_PREFIX); |
| 198 | + await closeConnections(); |
| 199 | + }); |
| 200 | + |
| 201 | + it('should process a <name> job from the queue', async () => { |
| 202 | + const job = await addJob(pg, databaseId, '<name>', { |
| 203 | + // Your test payload |
| 204 | + }); |
| 205 | + |
| 206 | + expect(job.id).toBeDefined(); |
| 207 | + const result = await waitForJobComplete(pg, job.id, { timeout: 30000 }); |
| 208 | + expect(['completed', 'failed']).toContain(result.status); |
| 209 | + }); |
| 210 | +}); |
| 211 | +``` |
| 212 | + |
| 213 | +**Important:** The e2e test filename must match the function name (`<name>.e2e.test.ts`) for the CI matrix to pick it up automatically. |
| 214 | + |
| 215 | +## Step 8: Test locally |
| 216 | + |
| 217 | +### Option A: Docker Compose + local Node (fastest iteration) |
| 218 | + |
| 219 | +```bash |
| 220 | +make dev # start postgres, mailpit, graphql-server |
| 221 | +pnpm dev:fn --only=<name> # run just your function |
| 222 | +``` |
| 223 | + |
| 224 | +### Option B: Skaffold (production-like k8s) |
| 225 | + |
| 226 | +```bash |
| 227 | +make skaffold-dev-<name> # deploys infra + just your function |
| 228 | +``` |
| 229 | + |
| 230 | +## Step 9: Verify CI will work |
| 231 | + |
| 232 | +The following CI workflows auto-discover functions — no manual edits needed: |
| 233 | + |
| 234 | +- **docker.yaml** — discovers `functions/*/handler.json`, builds Docker image per function |
| 235 | +- **test-k8s-deployment.yaml** — discovers functions with matching `*.e2e.test.ts`, runs per-function k8s e2e tests |
| 236 | +- **test.yaml** — runs `pnpm test:unit` which picks up your `__tests__/` directory |
| 237 | +- **ci.yaml** — runs `pnpm build` which builds your generated package |
| 238 | + |
| 239 | +## Checklist |
| 240 | + |
| 241 | +- [ ] `functions/<name>/handler.json` created with name, version, port |
| 242 | +- [ ] `functions/<name>/handler.ts` created with `FunctionHandler` export |
| 243 | +- [ ] `job/service/src/types.ts` — function name added to `FunctionName` union |
| 244 | +- [ ] `job/service/src/index.ts` — entry added to `functionRegistry` |
| 245 | +- [ ] `pnpm generate` ran successfully |
| 246 | +- [ ] `pnpm install && pnpm build` succeeds |
| 247 | +- [ ] Unit tests in `functions/<name>/__tests__/handler.test.ts` |
| 248 | +- [ ] E2e test in `tests/e2e/__tests__/<name>.e2e.test.ts` |
| 249 | +- [ ] `pnpm test:unit` passes |
| 250 | +- [ ] Local dev works (`make dev && pnpm dev:fn --only=<name>`) |
| 251 | +- [ ] No manual edits needed to skaffold.yaml, dev.ts, or job-service k8s config (all auto-generated) |
0 commit comments