Skip to content

Commit 433bddf

Browse files
Anmol1696Anmol1696
committed
ci(publish): add toolkit publish workflow + portable-functions doc
Wave 5a of the portable-functions toolkit. CI now publishes the six @constructive-io/fn-* packages to npm with provenance when an `fn-v*` tag is pushed. The workflow can also be triggered via workflow_dispatch with dry_run=true for pre-release verification. Publish order (deps first): fn-types → fn-app (knative-job-fn) → fn-runtime → fn-generator → fn-client → fn-cli Each package already has publishConfig.access=public and files[] set from its respective Wave 1/2/3 commit. Also adds docs/portable-functions-toolkit.md: package map, customer- repo flow, registry loader behaviour, release procedure, and a manual verification checklist for the first release. Documents the deferred follow-ups (Wave 4c k8s manifest migration, .ts config loader, fn init/dockerfile/k8s standalone subcommands, fn-templates packaging).
1 parent a0cfd7e commit 433bddf

2 files changed

Lines changed: 169 additions & 0 deletions

File tree

‎.github/workflows/publish.yaml‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
name: Publish toolkit packages
2+
3+
on:
4+
push:
5+
tags:
6+
- 'fn-v*' # toolkit release tag, e.g. fn-v0.1.0
7+
workflow_dispatch:
8+
inputs:
9+
dry_run:
10+
description: 'Run pnpm publish --dry-run only'
11+
type: boolean
12+
default: true
13+
14+
concurrency:
15+
group: publish-${{ github.ref }}
16+
cancel-in-progress: false
17+
18+
jobs:
19+
publish:
20+
name: Publish @constructive-io/fn-* to npm
21+
runs-on: ubuntu-latest
22+
permissions:
23+
contents: read
24+
id-token: write # required for npm provenance
25+
steps:
26+
- name: Checkout
27+
uses: actions/checkout@v5
28+
29+
- name: Setup pnpm
30+
uses: pnpm/action-setup@v6
31+
32+
- name: Setup Node.js
33+
uses: actions/setup-node@v5
34+
with:
35+
node-version: '22'
36+
registry-url: 'https://registry.npmjs.org'
37+
cache: 'pnpm'
38+
39+
- name: Generate function packages
40+
run: node --experimental-strip-types scripts/generate.ts
41+
42+
- name: Install dependencies
43+
run: pnpm install --frozen-lockfile
44+
45+
- name: Build toolkit packages
46+
run: |
47+
pnpm --filter @constructive-io/fn-types build
48+
pnpm --filter @constructive-io/knative-job-fn build
49+
pnpm --filter @constructive-io/fn-runtime build
50+
pnpm --filter @constructive-io/fn-generator build
51+
pnpm --filter @constructive-io/fn-client build
52+
pnpm --filter @constructive-io/fn-cli build
53+
54+
- name: Verify generator snapshot
55+
run: pnpm --filter @constructive-io/fn-generator test
56+
57+
- name: Publish (dry run for workflow_dispatch when requested)
58+
if: github.event_name == 'workflow_dispatch' && github.event.inputs.dry_run == 'true'
59+
run: |
60+
for pkg in fn-types fn-app fn-runtime fn-generator fn-client fn-cli; do
61+
(cd "packages/$pkg" && pnpm publish --dry-run --no-git-checks --access public)
62+
done
63+
64+
- name: Publish to npm with provenance
65+
if: github.event_name == 'push' || (github.event_name == 'workflow_dispatch' && github.event.inputs.dry_run != 'true')
66+
env:
67+
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
68+
NPM_CONFIG_PROVENANCE: 'true'
69+
run: |
70+
# Order matters: deps first, dependents last.
71+
for pkg in fn-types fn-app fn-runtime fn-generator fn-client fn-cli; do
72+
(cd "packages/$pkg" && pnpm publish --no-git-checks --access public)
73+
done

‎docs/portable-functions-toolkit.md‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
# Portable Functions Toolkit
2+
3+
The Constructive Functions toolkit lets any external repo `pnpm add` a small set of npm packages, drop in its own `functions/` directory, and get a full code-gen + Docker + k8s + manifest-registry pipeline — without git submodules or copy-paste.
4+
5+
## Package layout (Starship V2 style)
6+
7+
```
8+
fn-cli ──► fn-client ──► fn-generator ──► fn-types
9+
└────────────────────► fn-types
10+
└────────► fn-types
11+
fn-runtime ──► fn-types (handlers import this + fn-types)
12+
knative-job-fn (fn-app) (low-level Express middleware)
13+
```
14+
15+
| Package | Single responsibility |
16+
|---|---|
17+
| `@constructive-io/fn-types` | Source-of-truth TS types: `FunctionHandler`, `FunctionContext`, `HandlerManifest`, `FnRegistry`, `FnConfig` + `defineConfig()`. No logic. |
18+
| `@constructive-io/fn-runtime` | Express server factory + GraphQL clients + log + job-callback wiring. The contract handlers import. |
19+
| `@constructive-io/knative-job-fn` | Low-level Express middleware for Knative job request/response shape. fn-runtime depends on it. |
20+
| `@constructive-io/fn-generator` | Programmatic builders that emit Dockerfiles, k8s YAML, configmaps, skaffold profiles, manifest registry. Pure functions; idempotent file I/O at the boundary. |
21+
| `@constructive-io/fn-client` | Importable `FnClient` API — config loading, manifest reading, `pnpm build`, child-process orchestration for `dev`. |
22+
| `@constructive-io/fn-cli` | The `fn` executable. Subcommands: `generate`, `build`, `dev`, `manifest`, `verify`. |
23+
24+
## Customer repo experience
25+
26+
```
27+
my-app/
28+
├── functions/
29+
│ └── send-welcome/
30+
│ ├── handler.json # {"name":"send-welcome","version":"0.1.0","type":"node-graphql"}
31+
│ └── handler.ts # default-exported FunctionHandler
32+
├── fn.config.json # FnConfig (typed via fn-types)
33+
└── package.json
34+
```
35+
36+
```bash
37+
pnpm add -D @constructive-io/fn-cli
38+
pnpm add @constructive-io/fn-runtime
39+
40+
pnpm fn generate # write generated/<name>/ + manifest + skaffold
41+
pnpm fn build # pnpm -r build
42+
pnpm fn manifest # cat generated/functions-manifest.json
43+
pnpm fn verify # check manifest matches functions/
44+
pnpm fn dev # spawn each function as a Node child
45+
```
46+
47+
## Job-service registry (when running the `jobs-bundle` preset)
48+
49+
The job-service no longer hardcodes function names. It loads its registry at startup from one of three sources, in priority order:
50+
51+
1. `FUNCTIONS_REGISTRY` env var
52+
Format: `name:moduleName:port,...` — `moduleName` and `port` are optional (missing `moduleName` falls back to `@constructive-io/<name>-fn`).
53+
54+
2. `FUNCTIONS_MANIFEST_PATH` env var pointing to a JSON file with the existing `functions-manifest.json` shape. Manifest entries can carry an optional `moduleName` field; otherwise convention applies.
55+
56+
3. Default file: `<cwd>/generated/functions-manifest.json` — what `fn generate` produces.
57+
58+
Empty registry is allowed; lookups still throw `Unknown function "<name>"` to preserve the legacy behaviour.
59+
60+
## Releasing the toolkit (Wave 5)
61+
62+
The CI workflow at `.github/workflows/publish.yaml` publishes all six packages with [npm provenance](https://docs.npmjs.com/generating-provenance-statements) when a `fn-v*` tag is pushed. Steps:
63+
64+
1. Update versions in each `packages/fn-*/package.json` (and `packages/fn-app/package.json`). Bump in lock-step for now; we'll move to changesets later.
65+
2. Verify locally:
66+
```bash
67+
pnpm --filter '@constructive-io/fn-*' build
68+
pnpm --filter @constructive-io/fn-generator test
69+
pnpm --filter @constructive-io/fn-client test
70+
for pkg in fn-types fn-app fn-runtime fn-generator fn-client fn-cli; do
71+
(cd "packages/$pkg" && pnpm publish --dry-run --no-git-checks --access public)
72+
done
73+
```
74+
3. Tag and push:
75+
```bash
76+
git tag fn-v0.1.0
77+
git push origin fn-v0.1.0
78+
```
79+
4. CI publishes in dependency order: `fn-types` → `fn-app` → `fn-runtime` → `fn-generator` → `fn-client` → `fn-cli`.
80+
81+
You can also run the workflow with `workflow_dispatch` (default `dry_run: true`) to verify packing before tagging.
82+
83+
## Verification checklist (manual, before first release)
84+
85+
- [ ] **Snapshot regression**: `pnpm --filter @constructive-io/fn-generator test` passes (asserts byte-identical output vs `scripts/generate.ts`).
86+
- [ ] **Job-registry tests**: `pnpm exec jest tests/integration/job-registry.test.ts` — six cases pass.
87+
- [ ] **Brasilia E2E**: with the live k8s stack running (`make skaffold-dev`), `pnpm test:e2e` still picks up jobs end-to-end.
88+
- [ ] **Scratch repo**: in a fresh `/tmp/test-fn-app` repo, `pnpm add -D @constructive-io/fn-cli && pnpm add @constructive-io/fn-runtime`, add `functions/hello/handler.{json,ts}`, run `fn generate && fn build && fn manifest`. Confirm output is sensible and `docker build -f generated/hello/Dockerfile .` succeeds.
89+
- [ ] **Hub integration**: in `constructive-hub/istanbul`, `pnpm bootstrap && pnpm start` still launches `send-email-link` and processes a job (the hub does not yet consume the new toolkit; this confirms Wave 1-3 didn't regress the existing submodule path).
90+
91+
## Deferred follow-ups (not in this branch)
92+
93+
- **Wave 4c — replace hand-written `k8s/base/functions/*.yaml` with generator output**. The hand-written manifests carry mailgun secrets, dry-run env vars, and a different image strategy (single bundled image, args-driven entry vs per-function image with Dockerfile CMD). Migrating safely requires either teaching `KnativeServiceBuilder` to emit those fields or providing a Kustomize patch overlay. Tracked separately.
94+
- **fn.config.ts/.js loading** — JSON only for now. Adding `.ts` requires an `esbuild`/`jiti` loader.
95+
- **`fn init` and `fn dockerfile` / `fn k8s` standalone subcommands** — the underlying builders exist (`buildPackages`, `buildSkaffold`); these are thin CLI wrappers to add later.
96+
- **Templates packaging** — currently `templatesDir` is a constructor option pointing at the host repo's `templates/`. A future change can ship templates inside `fn-generator` (or a separate `fn-templates` package) so customer repos don't need their own copy.

0 commit comments

Comments
 (0)