feature: templates#13
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces an initial “function templating” system that reduces each function to handler.ts + handler.json and generates runnable workspace packages under generated/, backed by new shared runtime/app packages and updated job orchestration.
Changes:
- Add a generator (
scripts/generate.ts) that createsgenerated/<fn>/packages (package.json/tsconfig/index.ts) and symlinks handlers fromfunctions/. - Introduce shared packages:
@constructive-io/knative-job-fn(app/callback handling) and@constructive-io/fn-runtime(context + GraphQL clients + handler wiring). - Restructure workspace/config to use
generated/*, add job server/worker/service packages, and add local docker-compose + K8s overlay for running generated functions.
Reviewed changes
Copilot reviewed 52 out of 55 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/generate.ts | Generates generated/* workspace packages and symlinks handler sources. |
| pnpm-workspace.yaml | Switch workspace packages to generated/*, packages/*, job/*. |
| pnpm-lock.yaml | Lockfile updates for new workspace structure and packages. |
| packages/fn-runtime/tsconfig.json | TS build config for fn-runtime output + declarations. |
| packages/fn-runtime/src/types.ts | Defines handler/context/public runtime types. |
| packages/fn-runtime/src/server.ts | Wires handler into job app + context builder. |
| packages/fn-runtime/src/index.ts | Exports fn-runtime public API. |
| packages/fn-runtime/src/graphql.ts | Creates tenant/meta GraphQL clients with headers/env. |
| packages/fn-runtime/src/context.ts | Builds per-request context (job meta, logger, GraphQL clients/stubs). |
| packages/fn-runtime/package.json | Declares fn-runtime package and deps. |
| packages/fn-app/tsconfig.json | TS build config for job app package output + declarations. |
| packages/fn-app/src/index.ts | Implements Express job app wrapper + callback protocol. |
| packages/fn-app/package.json | Declares job app package and deps. |
| package.json | Adds generate and preinstall generator scripts. |
| k8s/overlays/local-simple/simple-email.yaml | Local overlay to run generated simple-email function container command. |
| k8s/overlays/local-simple/send-email-link.yaml | Local overlay to run generated send-email-link function container command. |
| k8s/overlays/local-simple/kustomization.yaml | Kustomize resources for local-simple overlay. |
| job/worker/tsconfig.json | TS build config for worker output + declarations. |
| job/worker/src/run.ts | CLI entrypoint for worker. |
| job/worker/src/req.ts | HTTP dispatch helper for worker → function gateway. |
| job/worker/src/index.ts | Worker implementation (poll/listen/dispatch jobs). |
| job/worker/package.json | Worker package definition + deps. |
| job/service/tsconfig.json | Enables declarations for job service package. |
| job/service/src/types.ts | Types for orchestrator configuration/result. |
| job/service/src/run.ts | CLI entrypoint for orchestrator boot. |
| job/service/src/index.ts | Orchestrator to run functions + jobs server/worker/scheduler. |
| job/service/package.json | Job service package definition + deps. |
| job/server/tsconfig.json | Enables declarations for job server package. |
| job/server/src/run.ts | CLI entrypoint for job callback server. |
| job/server/src/index.ts | Express callback server for job completion/failure. |
| job/server/package.json | Job server package definition + deps. |
| functions/simple-email/tsconfig.esm.json | Removes old per-function TS config. |
| functions/simple-email/src/index.ts | Removes old Express entrypoint implementation. |
| functions/simple-email/package.json | Removes old function package manifest (now generated). |
| functions/simple-email/handler.ts | New handler-only implementation for simple-email. |
| functions/simple-email/handler.json | New handler manifest (name/version/deps) for generator. |
| functions/simple-email/README.md | Removes old per-function README (templating docs moved elsewhere). |
| functions/simple-email/Dockerfile | Removes old per-function Dockerfile (now repo-level/dev flow). |
| functions/simple-email/CHANGELOG.md | Removes old per-function changelog. |
| functions/send-email-link/types.d.ts | Adds local type shim for @launchql/mjml. |
| functions/send-email-link/tsconfig.esm.json | Removes old per-function TS config. |
| functions/send-email-link/package.json | Removes old function package manifest (now generated). |
| functions/send-email-link/handler.ts | Refactors to handler-only form using fn-runtime context. |
| functions/send-email-link/handler.json | New handler manifest (name/version/deps) for generator. |
| functions/send-email-link/README.md | Removes old per-function README (templating docs moved elsewhere). |
| functions/send-email-link/Dockerfile | Removes old per-function Dockerfile. |
| functions/send-email-link/CHANGELOG.md | Removes old per-function changelog. |
| functions/example/handler.ts | Adds example handler for templating system. |
| functions/example/handler.json | Adds example handler manifest for generator. |
| docs/spec/function-templating.md | Documents the templating system, runtime, and workflows. |
| docker-compose.yml | Adds local compose workflow to run job-service + postgres. |
| Makefile | Updates targets for generate/dev via docker compose; removes old docker build/push targets. |
| Dockerfile.dev | Adds dev image build to run generator/install/build in container. |
| AGENTS.md | Updates agent guide to new templating architecture/workflows. |
| .gitignore | Ignores generated/ directory. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
Comments suppressed due to low confidence (1)
functions/send-email-link/handler.ts:278
handlerreturns an{ error: ... }object whendatabaseIdis missing, butfn-runtimewill still respond 200 andfn-appwill treat the invocation as a success (sending a success callback). If missingdatabaseIdshould fail the job, throw an error instead (or add a runtime-level convention to map{ error }results to non-2xx / X-Job-Error).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "engines": { | ||
| "node": ">=18.17.0" | ||
| }, | ||
| "packageManager": "pnpm@10.12.2", | ||
| "scripts": { | ||
| "generate": "node --experimental-strip-types scripts/generate.ts", | ||
| "preinstall": "node --experimental-strip-types scripts/generate.ts", |
There was a problem hiding this comment.
The new generate/preinstall scripts use node --experimental-strip-types, but engines.node still allows Node >=18. On Node versions that don’t support this flag, installs will fail. Either bump engines.node to the minimum Node version that supports --experimental-strip-types, or change the generator to run via a supported toolchain (e.g., precompiled JS, tsx, or a small JS wrapper).
| import requestLib from 'request'; | ||
| import { | ||
| getCallbackBaseUrl, | ||
| getJobGatewayConfig, | ||
| getJobGatewayDevMap, | ||
| getNodeEnvironment | ||
| } from '@constructive-io/job-utils'; | ||
| import { Logger } from '@pgpmjs/logger'; |
There was a problem hiding this comment.
This introduces the deprecated request package for HTTP calls. request is unmaintained and has a long history of security issues via transitive deps; consider switching to Node’s built-in fetch/undici (or another maintained client) to reduce risk and dependency weight.
| export { bootJobs, startJobsServices, waitForJobsPrereqs } from './index'; | ||
|
|
||
| import { bootJobs } from './index'; | ||
|
|
There was a problem hiding this comment.
run.ts both re-exports symbols from ./index and separately imports bootJobs from ./index. This is redundant and can make the entrypoint harder to follow; consider exporting from ./index only, or importing once and re-exporting explicitly.
| const fs = require('fs') as typeof import('fs'); | ||
| const path = require('path') as typeof import('path'); | ||
|
|
||
| const ROOT: string = process.cwd(); | ||
| const FUNCTIONS_DIR: string = path.resolve(ROOT, 'functions'); | ||
| const GENERATED_DIR: string = path.resolve(ROOT, 'generated'); | ||
|
|
||
| interface FunctionManifest { | ||
| name: string; | ||
| version: string; | ||
| description?: string; | ||
| dependencies?: Record<string, string>; |
There was a problem hiding this comment.
scripts/generate.ts claims it runs during preinstall to avoid compilation, but the script itself uses TypeScript-only syntax (as typeof import('fs'), type annotations, interfaces). This is only safe as long as all environments run Node with --experimental-strip-types; if that’s not guaranteed, consider generating a plain JS version of this script (or adding a build step) so installs don’t depend on experimental Node flags.
| (res) => { | ||
| // Drain response data but ignore contents; callback server | ||
| // only uses status for debugging. | ||
| res.on('data', () => {}); | ||
| res.on('end', () => resolve()); | ||
| } |
There was a problem hiding this comment.
postJson() resolves successfully for any HTTP status code because it ignores res.statusCode. If the callback endpoint responds with 4xx/5xx, the job callback will be treated as sent and no error will be logged, which can leave jobs stuck without visibility. Consider rejecting (or at least logging) on non-2xx status codes, and optionally capturing statusCode in logs.
# Conflicts: # functions/send-email-link/handler.ts # functions/send-email-link/package.json # functions/simple-email/package.json # functions/simple-email/src/index.ts # functions/simple-email/tsconfig.esm.json # package.json # packages/fn-runtime/tsconfig.json
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 44 out of 47 changed files in this pull request and generated 4 comments.
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
Comments suppressed due to low confidence (2)
functions/send-email-link/handler.ts:277
- When
databaseIdis missing, the handler returns{ error: ... }with a 200 response.createJobApp()treats 2xx responses as success and will emit a success callback, so this will incorrectly mark the job complete. Throw an error (or otherwise trigger the fn-app error middleware) whendatabaseIdis required.
functions/send-email-link/handler.ts:6 sendSmtpis still used later in this module (SMTP code path) but its import was removed in this change, which will cause a compile-time error. Re-add thesimple-smtp-serverimport or remove/guard the SMTP branch.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @@ -0,0 +1,6 @@ | |||
| { | |||
| "name": "knative-job-example", | |||
There was a problem hiding this comment.
handler.json.name is set to knative-job-example, but the generator uses the directory name (functions/example) for discovery/output and --only=<name> filtering. Because the template Dockerfile passes --only={{name}}, this mismatch will prevent generating/building the example function. Align name with the folder name (or change the generator/Dockerfile to consistently use one identifier).
| "name": "knative-job-example", | |
| "name": "functions/example", |
| const imageName = `${registry}/${manifest.name}-fn`; | ||
|
|
||
| console.log(`\nBuilding ${imageName}:${tag}...`); | ||
| execSync( | ||
| `docker build -t ${imageName}:${tag} -f ${dockerfilePath} .`, | ||
| { cwd: ROOT, stdio: 'inherit' } | ||
| ); |
There was a problem hiding this comment.
execSync() is invoked with a shell command built via string interpolation (registry, tag, and manifest.name). If any of these contain whitespace or shell metacharacters (e.g., from a malicious/accidental handler.json.name), it can break the command or lead to shell injection. Prefer spawnSync('docker', ['build', '-t', ${imageName}:${tag}, '-f', dockerfilePath, '.'], { ... }) and/or validate/whitelist allowed characters for image names/tags.
| | `package.json` | Placeholder replacement + deep merge of handler.json `dependencies` | | ||
| | `tsconfig.json` | Copied verbatim + `.d.ts` filenames appended to `include` | | ||
| | `index.ts` | Placeholder replacement only | | ||
| | `Dockerfile` | Used by `scripts/docker-build.ts` (not copied to generated/) | |
There was a problem hiding this comment.
This spec says the template Dockerfile is “not copied to generated/”, but scripts/generate.ts currently copies all template files (including Dockerfile), and scripts/docker-build.ts explicitly expects generated/<dir>/Dockerfile to exist. Update the spec to match the implementation (or change the generator/docker-build behavior to match the spec).
| | `Dockerfile` | Used by `scripts/docker-build.ts` (not copied to generated/) | | |
| | `Dockerfile` | Copied to `generated/<name>/Dockerfile` and used by `scripts/docker-build.ts` | |
| | Field | Required | Description | | ||
| |---|---|---| | ||
| | `name` | Yes | Function name (used in package name and k8s manifests) | | ||
| | `version` | Yes | Semver version | | ||
| | `description` | No | Description for package.json | | ||
| | `type` | No | Template type from `templates/<type>/` (default: `"node-graphql"`) | | ||
| | `dependencies` | No | Extra npm deps merged into template's base package.json | |
There was a problem hiding this comment.
The spec describes handler.json.name as the function name, but the generator also relies on the functions/
--only=<name> filtering. It would help to document that handler.json.name should match the folder name (or explicitly define which one is authoritative) to avoid mismatches that break generation/Docker builds.
Initial templatization of functions.
Built ontop of: #12