Skip to content

Commit c8e2728

Browse files
committed
feat(fn-generator): port scripts/generate.ts into a typed library
Wave 2 of the portable-functions toolkit. Introduces the programmatic generator that fn-client and fn-cli (next waves) will compose. @constructive-io/fn-generator@0.1.0 - FnGenerator class with discover()/buildPackages()/buildManifest()/ buildConfigMaps()/buildSkaffold()/apply()/generate() methods. - Pure builder layer (returns Manifest[]) + a single apply() boundary that does idempotent file I/O via writeIfChanged() and ensureSymlink(). - Builders: per-function package files (templates + shared + handler symlinks), functions-manifest.json, per-function and aggregate functions-configmap.yaml, root skaffold.yaml. - Honours --only and --packages-only modes (skip k8s/skaffold and per-function/aggregate configmaps respectively). - Auto-assigns ports starting at 8081, validates 8080 is reserved for the job-service, detects port conflicts. Verification: snapshot test runs FnGenerator against the brasilia repo's own functions/+templates/ into a tmp dir and asserts byte- identical output (file contents, symlink targets, skaffold.yaml) vs scripts/generate.ts. All three assertions pass. Also updates @constructive-io/fn-types FnRegistry to match the actual on-disk format ({name, dir, port, type}) and adds optional moduleName and url for the upcoming job-service rewrite (Wave 4). scripts/generate.ts is unchanged in this PR — Wave 4 will collapse it into a one-line shim that delegates to FnGenerator.
1 parent 4ee9472 commit c8e2728

17 files changed

Lines changed: 1835 additions & 5 deletions

packages/fn-generator/README.md

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# @constructive-io/fn-generator
2+
3+
Programmatic generator for the Constructive Functions toolkit. Given a `functions/<name>/handler.json` manifest and a set of templates, it produces:
4+
5+
- A workspace package per function (Dockerfile, entry point, package.json with merged deps, tsconfig, k8s YAML).
6+
- Symlinks back to the source `handler.{ts,py}` and any auxiliary `.d.ts` / `.py` files.
7+
- A `functions-manifest.json` registry.
8+
- Per-function and aggregate `functions-configmap.yaml` (Knative job-service registry).
9+
- A root `skaffold.yaml` with one profile per function plus an aggregate `local-simple` profile.
10+
11+
The library is **pure**: each builder returns a list of `Manifest` objects (`file` or `symlink`); `FnGenerator.apply()` is the only step that touches disk, and it does so idempotently (no rewrite if the on-disk content already matches).
12+
13+
## Usage
14+
15+
```ts
16+
import { FnGenerator } from '@constructive-io/fn-generator';
17+
18+
const generator = new FnGenerator({
19+
rootDir: process.cwd(), // default
20+
functionsDir: 'functions', // default <root>/functions
21+
outputDir: 'generated', // default <root>/generated
22+
templatesDir: 'templates', // default <root>/templates
23+
namespace: 'constructive-functions', // default
24+
});
25+
26+
// One-shot
27+
generator.generate(); // all functions
28+
generator.generate({ only: 'simple-email' }); // single
29+
generator.generate({ packagesOnly: true }); // skip k8s/skaffold
30+
31+
// Or assemble manifests yourself
32+
const fns = generator.discover();
33+
const manifests = [
34+
...generator.buildPackages(fns),
35+
generator.buildManifest(fns),
36+
...generator.buildConfigMaps(fns),
37+
generator.buildSkaffold(fns),
38+
];
39+
const result = generator.apply(manifests);
40+
console.log(result.filesWritten, result.symlinksCreated);
41+
```
42+
43+
## Byte-identical output guarantee
44+
45+
`FnGenerator` is a port of the legacy `scripts/generate.ts` and is verified by a snapshot regression test against that script's output. Any change that breaks byte-identicalness is a bug.
Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
import * as fs from 'fs';
2+
import * as path from 'path';
3+
import { FnGenerator } from '../src';
4+
5+
/**
6+
* Regression test: run FnGenerator against the brasilia repo's own
7+
* `functions/` and `templates/`, output to a temp dir, and assert it
8+
* is byte-identical to the brasilia repo's checked-in `generated/`
9+
* (which is produced by the legacy `scripts/generate.ts`).
10+
*/
11+
12+
const ROOT = path.resolve(__dirname, '..', '..', '..');
13+
const FUNCTIONS_DIR = path.join(ROOT, 'functions');
14+
const TEMPLATES_DIR = path.join(ROOT, 'templates');
15+
const BASELINE_GENERATED = path.join(ROOT, 'generated');
16+
const BASELINE_SKAFFOLD = path.join(ROOT, 'skaffold.yaml');
17+
18+
const walk = (dir: string): { files: string[]; symlinks: string[] } => {
19+
const files: string[] = [];
20+
const symlinks: string[] = [];
21+
const recurse = (current: string, base: string): void => {
22+
const entries = fs.readdirSync(current);
23+
for (const entry of entries) {
24+
// Skip pnpm-installed dirs that aren't generator output.
25+
if (entry === 'node_modules' || entry === 'dist') continue;
26+
const full = path.join(current, entry);
27+
const rel = base ? path.join(base, entry) : entry;
28+
const stat = fs.lstatSync(full);
29+
if (stat.isSymbolicLink()) {
30+
symlinks.push(rel);
31+
} else if (stat.isDirectory()) {
32+
recurse(full, rel);
33+
} else {
34+
files.push(rel);
35+
}
36+
}
37+
};
38+
recurse(dir, '');
39+
return { files, symlinks };
40+
};
41+
42+
describe('FnGenerator snapshot vs scripts/generate.ts', () => {
43+
let tmpDir: string;
44+
let tmpRoot: string;
45+
46+
beforeAll(() => {
47+
// The generator writes skaffold.yaml at the rootDir, so we mirror the
48+
// brasilia layout into a tmp tree (functions/, templates/ symlinked to
49+
// the real ones; outputDir in tmpDir; rootDir = tmpRoot).
50+
tmpRoot = fs.mkdtempSync(path.join(require('os').tmpdir(), 'fn-gen-test-'));
51+
tmpDir = path.join(tmpRoot, 'generated');
52+
53+
fs.symlinkSync(FUNCTIONS_DIR, path.join(tmpRoot, 'functions'));
54+
fs.symlinkSync(TEMPLATES_DIR, path.join(tmpRoot, 'templates'));
55+
56+
const gen = new FnGenerator({ rootDir: tmpRoot });
57+
gen.generate();
58+
});
59+
60+
afterAll(() => {
61+
fs.rmSync(tmpRoot, { recursive: true, force: true });
62+
});
63+
64+
it('produces the same files (byte-identical)', () => {
65+
const baseline = walk(BASELINE_GENERATED);
66+
const actual = walk(tmpDir);
67+
expect([...actual.files].sort()).toEqual([...baseline.files].sort());
68+
for (const rel of baseline.files) {
69+
const a = fs.readFileSync(path.join(tmpDir, rel), 'utf-8');
70+
const b = fs.readFileSync(path.join(BASELINE_GENERATED, rel), 'utf-8');
71+
expect({ file: rel, content: a }).toEqual({ file: rel, content: b });
72+
}
73+
});
74+
75+
it('produces the same symlinks (same relative target)', () => {
76+
const baseline = walk(BASELINE_GENERATED);
77+
const actual = walk(tmpDir);
78+
expect([...actual.symlinks].sort()).toEqual([...baseline.symlinks].sort());
79+
for (const rel of baseline.symlinks) {
80+
const a = fs.readlinkSync(path.join(tmpDir, rel));
81+
const b = fs.readlinkSync(path.join(BASELINE_GENERATED, rel));
82+
expect({ symlink: rel, target: a }).toEqual({ symlink: rel, target: b });
83+
}
84+
});
85+
86+
it('produces an identical skaffold.yaml', () => {
87+
const a = fs.readFileSync(path.join(tmpRoot, 'skaffold.yaml'), 'utf-8');
88+
const b = fs.readFileSync(BASELINE_SKAFFOLD, 'utf-8');
89+
expect(a).toBe(b);
90+
});
91+
});
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
module.exports = {
2+
preset: 'ts-jest',
3+
testEnvironment: 'node',
4+
testMatch: ['**/__tests__/**/*.test.ts'],
5+
collectCoverageFrom: ['src/**/*.ts'],
6+
};

packages/fn-generator/package.json

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
{
2+
"name": "@constructive-io/fn-generator",
3+
"version": "0.1.0",
4+
"description": "Programmatic generator for Constructive Functions: walks handler.json manifests and stamps out Dockerfiles, k8s YAML, configmaps, and Skaffold profiles. Pure functions; file I/O at the boundary.",
5+
"author": "Constructive <developers@constructive.io>",
6+
"license": "SEE LICENSE IN LICENSE",
7+
"main": "dist/index.js",
8+
"types": "dist/index.d.ts",
9+
"files": [
10+
"dist",
11+
"README.md"
12+
],
13+
"publishConfig": {
14+
"access": "public"
15+
},
16+
"scripts": {
17+
"build": "tsc -p tsconfig.json",
18+
"clean": "rimraf dist",
19+
"test": "jest"
20+
},
21+
"dependencies": {
22+
"@constructive-io/fn-types": "workspace:^"
23+
},
24+
"devDependencies": {
25+
"@types/jest": "^29.5.14",
26+
"@types/node": "^22.10.4",
27+
"jest": "^29.7.0",
28+
"rimraf": "^5.0.5",
29+
"ts-jest": "^29.2.5",
30+
"typescript": "^5.1.6"
31+
}
32+
}
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
import * as fs from 'fs';
2+
import * as path from 'path';
3+
import { renderTemplate } from '../placeholders';
4+
import type { FunctionInfo, Manifest } from '../types';
5+
6+
/**
7+
* Build a `functions-configmap.yaml` for either a single function (when
8+
* `target` is supplied) or the aggregate of all functions.
9+
*
10+
* - per-function output: `<outputDir>/<target.dir>/k8s/functions-configmap.yaml`
11+
* - aggregate output: `<outputDir>/functions-configmap.yaml`
12+
*/
13+
export const buildConfigMap = (args: {
14+
fns: FunctionInfo[];
15+
target?: FunctionInfo;
16+
outputDir: string;
17+
templatesDir: string;
18+
namespace: string;
19+
}): Manifest => {
20+
const targetFns = args.target ? [args.target] : args.fns;
21+
const gatewayMap: Record<string, string> = {};
22+
for (const fn of targetFns) {
23+
gatewayMap[fn.name] = `http://${fn.name}.${args.namespace}.svc.cluster.local`;
24+
}
25+
26+
const template = fs.readFileSync(
27+
path.join(args.templatesDir, 'k8s', 'functions-configmap.yaml'),
28+
'utf-8'
29+
);
30+
const yaml = renderTemplate(template, {
31+
jobs_supported: targetFns.map((fn) => fn.name).join(','),
32+
gateway_map: JSON.stringify(gatewayMap),
33+
});
34+
35+
const outPath = args.target
36+
? path.join(args.outputDir, args.target.dir, 'k8s', 'functions-configmap.yaml')
37+
: path.join(args.outputDir, 'functions-configmap.yaml');
38+
39+
return { kind: 'file', path: outPath, content: yaml };
40+
};
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
import * as path from 'path';
2+
import type { FunctionInfo, Manifest } from '../types';
3+
4+
/**
5+
* Build the `generated/functions-manifest.json` manifest. Field order is
6+
* fixed (`name`, `dir`, `port`, `type`) to keep output byte-identical.
7+
*/
8+
export const buildManifestJson = (
9+
fns: FunctionInfo[],
10+
outputDir: string
11+
): Manifest => {
12+
const data = {
13+
functions: fns.map((fn) => ({
14+
name: fn.name,
15+
dir: fn.dir,
16+
port: fn.port,
17+
type: fn.type,
18+
})),
19+
};
20+
return {
21+
kind: 'file',
22+
path: path.join(outputDir, 'functions-manifest.json'),
23+
content: JSON.stringify(data, null, 2) + '\n',
24+
};
25+
};
Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
import * as fs from 'fs';
2+
import * as path from 'path';
3+
import { walkTemplateFiles } from '../fs-utils';
4+
import { processTemplateFile } from '../placeholders';
5+
import type { FunctionInfo, Manifest } from '../types';
6+
7+
/**
8+
* Build all manifests for a single function:
9+
* - per-template files (placeholders applied, package.json/tsconfig.json processed)
10+
* - shared template files (same processing)
11+
* - symlinks for handler.{ts,py}, *.d.ts, and any other *.py
12+
*
13+
* `templateDir` is the resolved type-specific dir (`templates/node-graphql/` etc.).
14+
*/
15+
export const buildPackageManifests = (
16+
fn: FunctionInfo,
17+
args: {
18+
fnDir: string; // absolute: <functionsDir>/<dir>
19+
genDir: string; // absolute: <outputDir>/<dir>
20+
templateDir: string; // absolute: <templatesDir>/<type>
21+
sharedDir?: string; // absolute: <templatesDir>/shared (if present)
22+
}
23+
): Manifest[] => {
24+
const out: Manifest[] = [];
25+
26+
// 1. Per-template files
27+
const templateFiles = walkTemplateFiles(args.templateDir);
28+
for (const relPath of templateFiles) {
29+
const templateFile = path.join(args.templateDir, relPath);
30+
const outputFile = path.join(args.genDir, relPath);
31+
const templateContent = fs.readFileSync(templateFile, 'utf-8');
32+
const baseName = path.basename(relPath);
33+
const processed = processTemplateFile(baseName, templateContent, fn.manifest, args.fnDir);
34+
out.push({ kind: 'file', path: outputFile, content: processed });
35+
}
36+
37+
// 2. Shared template files
38+
if (args.sharedDir && fs.existsSync(args.sharedDir)) {
39+
const sharedFiles = walkTemplateFiles(args.sharedDir);
40+
for (const relPath of sharedFiles) {
41+
const templateFile = path.join(args.sharedDir, relPath);
42+
const outputFile = path.join(args.genDir, relPath);
43+
const templateContent = fs.readFileSync(templateFile, 'utf-8');
44+
const baseName = path.basename(relPath);
45+
const processed = processTemplateFile(baseName, templateContent, fn.manifest, args.fnDir);
46+
out.push({ kind: 'file', path: outputFile, content: processed });
47+
}
48+
}
49+
50+
// 3. Handler symlink (handler.ts wins; handler.py only if no handler.ts)
51+
const handlerTs = path.join(args.fnDir, 'handler.ts');
52+
const handlerPy = path.join(args.fnDir, 'handler.py');
53+
if (fs.existsSync(handlerTs)) {
54+
out.push({ kind: 'symlink', path: path.join(args.genDir, 'handler.ts'), target: handlerTs });
55+
} else if (fs.existsSync(handlerPy)) {
56+
out.push({ kind: 'symlink', path: path.join(args.genDir, 'handler.py'), target: handlerPy });
57+
}
58+
59+
// 4. Auxiliary symlinks: all *.d.ts and *.py (except handler.py already linked above)
60+
const files = fs.readdirSync(args.fnDir);
61+
for (const file of files) {
62+
if (file.endsWith('.d.ts') || (file.endsWith('.py') && file !== 'handler.py')) {
63+
out.push({
64+
kind: 'symlink',
65+
path: path.join(args.genDir, file),
66+
target: path.join(args.fnDir, file),
67+
});
68+
}
69+
}
70+
71+
return out;
72+
};
Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
import * as fs from 'fs';
2+
import * as path from 'path';
3+
import { renderTemplate } from '../placeholders';
4+
import type { FunctionInfo, Manifest } from '../types';
5+
6+
const PYTHON_ARTIFACTS = ` - image: constructive-functions-python
7+
context: .
8+
docker:
9+
dockerfile: Dockerfile.python.dev
10+
sync:
11+
manual:
12+
- src: 'functions/**/*.py'
13+
dest: /usr/src/app
14+
- src: 'generated/**/*.py'
15+
dest: /usr/src/app`;
16+
17+
/**
18+
* Build the root `skaffold.yaml` from `templates/k8s/skaffold.yaml`,
19+
* including per-function profiles, an aggregate raw-yaml list, port
20+
* forwards, and the Python build artifact (only if any function is python).
21+
*/
22+
export const buildSkaffold = (args: {
23+
fns: FunctionInfo[];
24+
rootDir: string;
25+
templatesDir: string;
26+
namespace: string;
27+
}): Manifest => {
28+
const k8s = path.join(args.templatesDir, 'k8s');
29+
const nodeProfile = fs.readFileSync(path.join(k8s, 'skaffold-profile.yaml'), 'utf-8');
30+
const pythonProfile = fs.readFileSync(path.join(k8s, 'skaffold-profile-python.yaml'), 'utf-8');
31+
const main = fs.readFileSync(path.join(k8s, 'skaffold.yaml'), 'utf-8');
32+
33+
const perFnProfiles = args.fns
34+
.map((fn) =>
35+
renderTemplate(fn.type === 'python' ? pythonProfile : nodeProfile, {
36+
name: fn.name,
37+
dir: fn.dir,
38+
port: String(fn.port),
39+
namespace: args.namespace,
40+
}).trimEnd()
41+
)
42+
.join('\n');
43+
44+
const allRawYaml = args.fns
45+
.map((fn) => ` - generated/${fn.dir}/k8s/local-deployment.yaml`)
46+
.join('\n');
47+
48+
const allPortForwards = args.fns
49+
.map((fn) =>
50+
[
51+
' - resourceType: service',
52+
` resourceName: ${fn.name}`,
53+
` namespace: ${args.namespace}`,
54+
' port: 80',
55+
` localPort: ${fn.port}`,
56+
].join('\n')
57+
)
58+
.join('\n');
59+
60+
const pythonArtifacts = args.fns.some((fn) => fn.type === 'python') ? PYTHON_ARTIFACTS : '';
61+
62+
const skaffold = renderTemplate(main, {
63+
per_function_profiles: perFnProfiles,
64+
all_raw_yaml: allRawYaml,
65+
all_port_forwards: allPortForwards,
66+
python_artifacts: pythonArtifacts,
67+
namespace: args.namespace,
68+
});
69+
70+
return { kind: 'file', path: path.join(args.rootDir, 'skaffold.yaml'), content: skaffold };
71+
};

0 commit comments

Comments
 (0)