Feature/cleanup email functions#9
Conversation
functions testing harness
Dev/testing strategy using a matrix
|
Only thing is, we might want to keep the changes in a seperate branch. |
change this pr to draft. done |
| @@ -1,17 +1,13 @@ | |||
| FROM node:22-alpine | |||
| FROM node:20-alpine | |||
There was a problem hiding this comment.
we should not do this though.
|
|
||
| COPY dist ./dist | ||
|
|
||
| ENV NODE_ENV=production | ||
| ENV PORT=8080 | ||
|
|
||
| USER node | ||
|
|
||
| CMD ["node", "dist/index.js"] |
There was a problem hiding this comment.
the docker image, does this build?
lets add a way to build the docker image in github actions
| @@ -1,23 +1,15 @@ | |||
|
|
|||
| import app from '@constructive-io/knative-job-fn'; | |||
There was a problem hiding this comment.
functions need to be in functions/ dir
| @@ -1,90 +0,0 @@ | |||
| .PHONY: build clean lint test test-all build-test-runner docker-build docker-build-simple-email docker-build-send-email-link docker-push docker-push-simple-email docker-push-send-email-link | |||
| @@ -1,96 +0,0 @@ | |||
| # Constructive Functions Agent Guide | |||
There was a problem hiding this comment.
lets keep this, but update it.
| @@ -1,83 +0,0 @@ | |||
| # Interweb / Constructive Kubernetes Setup | |||
| @@ -1,42 +0,0 @@ | |||
| apiVersion: serving.knative.dev/v1 | |||
There was a problem hiding this comment.
here we should have just 2 functions: send-email-link and simple-email.
rest we can remove
- Remove crypto-login, github-repo-creator, hello-world, llm-external, llm-internal-calvin, opencode-headless, pgpm-dump, pytorch-gpu, runtime-script, rust-hello-world, simple-bash, stripe-function, twilio-sms and test-utils - Remove simple-email/node_modules_bad - Remove corresponding k8s base function manifests Co-authored-by: Cursor <cursoragent@cursor.com>
- send-email-link, simple-email: Dockerfile build from repo root, copy _runtimes/node/runner.js, CMD runner.js dist/index.js - Add express and body-parser to both package.json for runner HTTP server Co-authored-by: Cursor <cursoragent@cursor.com>
…l only - Makefile: docker-build/push only for send-email-link and simple-email - k8s overlays/local: remove hello-world, runtime-script refs; list only send-email-link and simple-email in JOBS_SUPPORTED and dev map - scaffold-function.sh: use simple-email as node template, set TARGET_FILE Co-authored-by: Cursor <cursoragent@cursor.com>
- Add ci-lint-build.yaml: trigger on functions/send-email-link/**, functions/simple-email/**, root package config; run lint + build + test - test-k8s-deployment: matrix only send-email-link and simple-email Co-authored-by: Cursor <cursoragent@cursor.com>
- README: node run, Docker dry-run, GHCR image names, canonical source - .gitignore: node_modules_bad - package.json: build:docker -> make docker-build - pnpm-lock: update for send-email-link and simple-email deps Co-authored-by: Cursor <cursoragent@cursor.com>
da47323 to
acddbd2
Compare
There was a problem hiding this comment.
Pull request overview
This PR streamlines the repo to focus on the two email-related cloud functions (simple-email, send-email-link), removing a large set of unused/experimental functions, simplifying K8s/local config accordingly, and updating Docker build/CI/docs to match the new focus.
Changes:
- Removed many legacy function implementations, tests, and K8s function manifests; updated local overlay/job-service configuration to only include email functions.
- Updated Docker build flow to build from repo root (so images can include the shared Node runner) and adjusted Makefile/package scripts accordingly.
- Added a targeted GitHub Actions workflow to lint/build (and optionally test) the two email functions; updated README with local/Docker run guidance.
Reviewed changes
Copilot reviewed 108 out of 112 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| package.json | Renames docker build script to call make docker-build. |
| k8s/scripts/scaffold-function.sh | Switches default node template to simple-email and adjusts scaffolding logic. |
| k8s/overlays/local/kustomization.yaml | Removes local overlay resources for deleted functions. |
| k8s/overlays/local/constructive/knative-job-service.yaml | Restricts supported jobs/dev map to the two email functions. |
| k8s/base/functions/stripe-function.yaml | Removes obsolete function manifest. |
| k8s/base/functions/simple-bash.yaml | Removes obsolete function manifest. |
| k8s/base/functions/rust-hello-world.yaml | Removes obsolete function manifest. |
| k8s/base/functions/runtime-script.yaml | Removes obsolete function manifest. |
| k8s/base/functions/pytorch-gpu.yaml | Removes obsolete function manifest. |
| k8s/base/functions/opencode-headless.yaml | Removes obsolete function manifest. |
| k8s/base/functions/llm-internal-calvin.yaml | Removes obsolete function manifest. |
| k8s/base/functions/llm-external.yaml | Removes obsolete function manifest. |
| k8s/base/functions/hello-world.yaml | Removes obsolete function manifest. |
| k8s/base/functions/github-repo-creator.yaml | Removes obsolete function manifest. |
| k8s/base/functions/crypto-login.yaml | Removes obsolete function manifest. |
| functions/twilio-sms/tsconfig.json | Removes deprecated function code. |
| functions/twilio-sms/src/index.ts | Removes deprecated function code. |
| functions/twilio-sms/package.json | Removes deprecated function code. |
| functions/twilio-sms/tests/index.test.ts | Removes deprecated function tests. |
| functions/test-utils.ts | Removes shared test utility used by deleted functions. |
| functions/stripe-function/tsconfig.json | Removes deprecated function code. |
| functions/stripe-function/src/index.ts | Removes deprecated function code. |
| functions/stripe-function/package.json | Removes deprecated function code. |
| functions/stripe-function/tests/index.test.ts | Removes deprecated function tests. |
| functions/simple-email/package.json | Adds runtime deps needed by shared runner (express, body-parser). |
| functions/simple-email/node_modules_bad/typescript | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/ts-jest | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/rimraf | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/pgsql-test | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/kubernetesjs | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/jest | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/@types/node | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/@types/jest | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/@pgpmjs/env | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/@launchql/postmaster | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/@constructive-io/knative-job-fn | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/.pnpm-workspace-state-v1.json | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/.modules.yaml | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/.bin/tsserver | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/.bin/tsc | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/.bin/ts-jest | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/.bin/rimraf | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/.bin/jest | Removes accidentally committed install artifacts. |
| functions/simple-email/node_modules_bad/.bin/browserslist | Removes accidentally committed install artifacts. |
| functions/simple-email/Dockerfile | Builds from repo root and starts via shared runner.js. |
| functions/simple-bash/tsconfig.json | Removes deprecated function code. |
| functions/simple-bash/src/index.sh | Removes deprecated function code. |
| functions/simple-bash/package.json | Removes deprecated function code. |
| functions/simple-bash/jest.config.js | Removes deprecated function config/tests. |
| functions/simple-bash/tests/index.test.ts | Removes deprecated function tests. |
| functions/simple-bash/Makefile | Removes deprecated function tooling. |
| functions/simple-bash/Dockerfile.test | Removes deprecated function tooling. |
| functions/simple-bash/Dockerfile | Removes deprecated function tooling. |
| functions/send-email-link/package.json | Adds runtime deps needed by shared runner (express, body-parser). |
| functions/send-email-link/Dockerfile | Builds from repo root and starts via shared runner.js. |
| functions/rust-hello-world/src/main.rs | Removes deprecated function code. |
| functions/rust-hello-world/tests/index.test.ts | Removes deprecated function tests. |
| functions/rust-hello-world/Dockerfile | Removes deprecated function tooling. |
| functions/rust-hello-world/Cargo.toml | Removes deprecated function tooling. |
| functions/runtime-script/tsconfig.json | Removes deprecated function code. |
| functions/runtime-script/src/index.ts | Removes deprecated function code. |
| functions/runtime-script/package.json | Removes deprecated function code. |
| functions/runtime-script/jest.config.js | Removes deprecated function config/tests. |
| functions/runtime-script/tests/index.test.ts | Removes deprecated function tests. |
| functions/runtime-script/tests/app_jobs_schema.sql | Removes deprecated test fixture. |
| functions/runtime-script/Makefile | Removes deprecated function tooling. |
| functions/pytorch-gpu/src/main.py | Removes deprecated function code. |
| functions/pytorch-gpu/tests/index.test.ts | Removes deprecated function tests. |
| functions/pytorch-gpu/Dockerfile | Removes deprecated function tooling. |
| functions/pgpm-dump/tsconfig.json | Removes deprecated function code. |
| functions/pgpm-dump/src/index.ts | Removes deprecated function code. |
| functions/pgpm-dump/package.json | Removes deprecated function code. |
| functions/pgpm-dump/tests/run-k8s.sh | Removes deprecated function tests/tooling. |
| functions/pgpm-dump/tests/index.test.ts | Removes deprecated function tests. |
| functions/pgpm-dump/Makefile | Removes deprecated function tooling. |
| functions/opencode-headless/tsconfig.json | Removes deprecated function code. |
| functions/opencode-headless/src/index.ts | Removes deprecated function code. |
| functions/opencode-headless/scripts/build-calvin.sh | Removes deprecated build script. |
| functions/opencode-headless/package.json | Removes deprecated function code. |
| functions/opencode-headless/tests/run-k8s.sh | Removes deprecated function tests/tooling. |
| functions/opencode-headless/tests/index.test.ts | Removes deprecated function tests. |
| functions/llm-internal-calvin/tsconfig.json | Removes deprecated function code. |
| functions/llm-internal-calvin/src/index.ts | Removes deprecated function code. |
| functions/llm-internal-calvin/package.json | Removes deprecated function code. |
| functions/llm-internal-calvin/tests/run-k8s.sh | Removes deprecated function tests/tooling. |
| functions/llm-internal-calvin/tests/index.test.ts | Removes deprecated function tests. |
| functions/llm-external/tsconfig.json | Removes deprecated function code. |
| functions/llm-external/src/index.ts | Removes deprecated function code. |
| functions/llm-external/package.json | Removes deprecated function code. |
| functions/llm-external/tests/index.test.ts | Removes deprecated function tests. |
| functions/hello-world/tsconfig.json | Removes deprecated function code. |
| functions/hello-world/src/index.ts | Removes deprecated function code. |
| functions/hello-world/package.json | Removes deprecated function code. |
| functions/hello-world/tests/run-k8s.sh | Removes deprecated function tests/tooling. |
| functions/hello-world/tests/index.test.ts | Removes deprecated function tests. |
| functions/hello-world/Makefile | Removes deprecated function tooling. |
| functions/github-repo-creator/tsconfig.json | Removes deprecated function code. |
| functions/github-repo-creator/src/index.ts | Removes deprecated function code. |
| functions/github-repo-creator/package.json | Removes deprecated function code. |
| functions/github-repo-creator/tests/index.test.ts | Removes deprecated function tests. |
| functions/crypto-login/tsconfig.json | Removes deprecated function code. |
| functions/crypto-login/src/index.ts | Removes deprecated function code. |
| functions/crypto-login/package.json | Removes deprecated function code. |
| functions/crypto-login/tests/index.test.ts | Removes deprecated function tests. |
| README.md | Adds local run + Docker run documentation and CI/CD notes. |
| Makefile | Limits to email functions and updates docker build/push targets for root-context builds. |
| .gitignore | Adds ignore for **/node_modules_bad/. |
| .github/workflows/test-k8s-deployment.yaml | Narrows CI K8s test matrix to the two email functions. |
| .github/workflows/ci-lint-build.yaml | Adds targeted lint/build/test workflow for email functions. |
Files not reviewed (1)
- functions/simple-bash/pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| COPY functions/simple-email/package.json ./ | ||
|
|
||
| RUN npm install -g pnpm@10.12.2 && pnpm install --prod | ||
|
|
||
| COPY dist ./dist | ||
| COPY functions/simple-email/dist ./dist | ||
| COPY functions/_runtimes/node/runner.js ./ |
There was a problem hiding this comment.
The image installs dependencies from package.json only and does not copy pnpm-lock.yaml, so pnpm install --prod won’t be reproducible (and may unexpectedly change over time, especially with latest dependencies). Consider copying the lockfile into the build context and using pnpm install --frozen-lockfile --prod to make Docker builds deterministic.
| if [ "$TEMPLATE" == "bash" ]; then | ||
| TEMPLATE_FILE="$BASE_DIR/simple-bash.yaml" | ||
| SOURCE_TEMPLATE_DIR="$BASE_DIR/../../functions/simple-bash" | ||
| echo "Using BASH template..." | ||
| else | ||
| TEMPLATE_FILE="$BASE_DIR/hello-world.yaml" | ||
| SOURCE_TEMPLATE_DIR="$BASE_DIR/../../functions/hello-world" | ||
| echo "Using NODE template..." | ||
| TEMPLATE_FILE="$BASE_DIR/simple-email.yaml" | ||
| SOURCE_TEMPLATE_DIR="$BASE_DIR/../../functions/simple-email" | ||
| echo "Using NODE template (simple-email)..." |
There was a problem hiding this comment.
--template=bash scaffolding is currently broken: it points to k8s/base/functions/simple-bash.yaml and functions/simple-bash, but both were removed in this PR. Either remove the bash template option/branch, or restore/replace the bash template files so the script can succeed.
| # Replace template name with the new function name in the new file | ||
| sed -i.bak "s/simple-email/$FUNCTION_NAME/g" "$TARGET_FILE" |
There was a problem hiding this comment.
The YAML placeholder replacement is hard-coded to simple-email, so if --template=bash is selected the generated manifest/package name replacement will be wrong (and currently no-op if the placeholder differs). Consider replacing based on the chosen template (e.g., simple-email vs simple-bash) or using a single explicit placeholder token in templates (like __FUNCTION_NAME__).
| pnpm install | ||
| pnpm --filter "@constructive-io/send-email-link-fn" run build | ||
| # Required env (e.g. for send-email-link): GRAPHQL_URL, SEND_EMAIL_LINK_DRY_RUN=true, PORT=8080 | ||
| node functions/send-email-link/dist/index.js |
There was a problem hiding this comment.
Running node functions/send-email-link/dist/index.js (and similar) won’t start an HTTP server—the function modules only export a handler and rely on functions/_runtimes/node/runner.js (or pnpm --filter … start) to serve requests. Update the local-run instructions to invoke the runner, otherwise the process will exit immediately.
| node functions/send-email-link/dist/index.js | |
| pnpm --filter "@constructive-io/send-email-link-fn" start |
| **Docker run (minimal, dry-run)** — the image validates Mailgun env at startup even when not sending; use placeholders to bring the container up for testing. Run detached (`-d`) so you can curl from the same host; use `docker logs <container_id>` if the container exits. | ||
|
|
||
| ```bash | ||
| docker run -d -p 8080:8080 --name send-email-link-test \ | ||
| -e SEND_EMAIL_LINK_DRY_RUN=true \ | ||
| -e MAILGUN_DOMAIN=local.example.com \ | ||
| -e MAILGUN_FROM=noreply@local.example.com \ | ||
| -e MAILGUN_REPLY=reply@local.example.com \ | ||
| -e MAILGUN_KEY=placeholder-key \ | ||
| -e MAILGUN_API_KEY=placeholder-api-key \ | ||
| -e GRAPHQL_URL=http://host.docker.internal:3000/graphql \ | ||
| ghcr.io/constructive-io/constructive-functions/send-email-link:latest | ||
|
|
||
| # Then: | ||
| curl -X POST http://localhost:8080/ -H "Content-Type: application/json" -d '{}' | ||
| # Expected: JSON body like {"error":"Missing X-Database-Id header or DEFAULT_DATABASE_ID"} (400) — means the server is up. | ||
| # With header: curl -X POST http://localhost:8080/ -H "Content-Type: application/json" -H "X-Database-Id: <uuid>" -d '{"email_type":"invite_email","email":"a@b.com","invite_token":"x","sender_id":"<uuid>"}' | ||
| docker logs send-email-link-test # if something fails | ||
| docker stop send-email-link-test && docker rm send-email-link-test |
There was a problem hiding this comment.
The Docker run example sets GRAPHQL_URL, but the shared Node runner constructs context.client from GRAPHQL_ENDPOINT (defaulting to an in-cluster hostname). In a local docker run, this likely breaks send-email-link because it uses context.client for GraphQL queries without guarding all calls. Either document that GRAPHQL_ENDPOINT must be set to the same value as GRAPHQL_URL, or update the runtime/entrypoint to derive the endpoint from GRAPHQL_URL so the example works as written.
| - name: Lint (send-email-link, simple-email) | ||
| run: | | ||
| pnpm --filter "@constructive-io/send-email-link-fn" --filter "@constructive-io/simple-email-fn" run lint | ||
| continue-on-error: true # Required until eslint config (e.g. root flat config) is available for subpackages | ||
|
|
||
| - name: Build (send-email-link, simple-email) | ||
| run: | | ||
| pnpm --filter "@constructive-io/send-email-link-fn" --filter "@constructive-io/simple-email-fn" run build | ||
|
|
||
| - name: Test (send-email-link, simple-email) | ||
| run: | | ||
| pnpm --filter "@constructive-io/send-email-link-fn" --filter "@constructive-io/simple-email-fn" run test | ||
| continue-on-error: true |
There was a problem hiding this comment.
This workflow’s header comment says it “ensures” PRs pass lint/build, but lint is marked continue-on-error: true (and test too). If the intent is gating quality for these packages, remove continue-on-error (or downgrade the wording/name so it’s clear lint/test failures won’t block merges).
| # Copy only package.json first for better layer caching | ||
| COPY functions/send-email-link/package.json ./ | ||
|
|
||
| RUN npm install -g pnpm@10.12.2 && pnpm install --prod | ||
|
|
||
| COPY dist ./dist | ||
| COPY functions/send-email-link/dist ./dist | ||
| COPY functions/_runtimes/node/runner.js ./ |
There was a problem hiding this comment.
The image installs dependencies from package.json only and does not copy pnpm-lock.yaml, so pnpm install --prod won’t be reproducible (and may unexpectedly change over time, especially with latest dependencies). Consider copying the lockfile into the build context and using pnpm install --frozen-lockfile --prod to make Docker builds deterministic.
前因: CI Test K8s 失败 403 — 测试在 Pod 内跑 Jest,用 default SA 调 API 创建
第二个 Job 时无权限 (jobs.batch is forbidden)。
后果: 新增 test-runner-rbac.yaml,Role 授予 default SA 对 batch/jobs 与
pods、pods/log 的 create/get/list/delete;CI overlay 引用该资源。
Co-authored-by: Cursor <cursoragent@cursor.com>
前因: Copilot 指出未拷贝 lockfile,镜像内 pnpm install 非确定性。
后果: 两函数 Dockerfile 均拷贝 pnpm-lock.yaml、pnpm-workspace.yaml 与
对应 package.json;install 改为 --frozen-lockfile --prod --filter
"@constructive-io/<fn>-fn",保持 dist 与 runner 路径不变。
Co-authored-by: Cursor <cursoragent@cursor.com>
前因: simple-bash 已删除,--template=bash 会指向不存在模板。
后果: 移除 bash 分支,仅支持 --template=node;传入 bash 时打印错误并退出。
Usage 文案改为 [--template=node]。
Co-authored-by: Cursor <cursoragent@cursor.com>
前因: 直接 node dist/index.js 不启动 HTTP;runner 用 GRAPHQL_ENDPOINT,
文档只写 GRAPHQL_URL 易混淆。
后果: 本地运行改为两种方式:pnpm --filter "..." start 或
node functions/_runtimes/node/runner.js functions/<name>/dist/index.js;
Docker 示例补 GRAPHQL_ENDPOINT 与 runner/function 变量说明。
Co-authored-by: Cursor <cursoragent@cursor.com>
前因: workflow 描述为「确保通过」但 lint/test 使用 continue-on-error,
表述与行为不符。
后果: 名称改为 CI Lint & Build (lint/test non-blocking);注释与步骤名
加 [non-blocking],明确仅 build 必须通过。
Co-authored-by: Cursor <cursoragent@cursor.com>
前因: 需要本地复现 CI 中 Test K8s 的步骤以便调试。
后果: 新增 RUN-CI-LOCALLY.md,说明 kind 集群、build/push 镜像、
kustomize 与运行 test-runner 的完整命令与注意事项。
Co-authored-by: Cursor <cursoragent@cursor.com>
|
Closing this for now. |
Feature/cleanup cloud functions