Skip to content

feat: worker dynamic secret/resource resolution for resource association#107

Closed
pyramation wants to merge 3 commits into
developfrom
feat/function-resource-resolution
Closed

feat: worker dynamic secret/resource resolution for resource association#107
pyramation wants to merge 3 commits into
developfrom
feat/function-resource-resolution

Conversation

@pyramation

@pyramation pyramation commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds runtime support for dynamic secret and resource resolution in the compute worker, enabling functions to declare required_secrets/required_configs in their definitions and have them validated from the DB at dispatch time.

Core changes:

  1. SecretsLoader (packages/module-loader/src/secrets-loader.ts) — new domain loader that queries metaschema_modules_public.config_secrets_module to discover the secrets table, then resolves secrets by name + namespace. TTL-cached, multi-database aware, follows the same pattern as BillingLoader/ComputeModuleLoader.

  2. ComputeWorker.validateSecrets() — called in doWork() after function discovery but before dispatch. Checks that declared required_secrets/required_configs exist in the config_secrets_module; logs warnings for missing entries. Does not inject into process.env — actual secret delivery to Knative services is handled by the provisioning pipeline via K8s secrets.

  3. FunctionContext extended with optional namespace, buckets, apis fields for future resource association phases. Secrets continue to flow via context.env (populated by Knative env vars, not worker injection).

  4. Three stub function handlers for the provisioning pipeline (validate + log only, no DB calls):

    • namespace-provision — validates namespace_name format
    • function-provision — validates function_id, logs service_url and secret count
    • function-sync-resources — validates resource declarations, logs summary
// Resolution flow in ComputeWorker.doWork():
fn = discovery.resolve(taskIdentifier)  // step 2: cached DB lookup
await validateSecrets(fn, dbId)         // NEW: check existence, warn on missing
await doWorkHttp(job, fn, ...)          // step 4: dispatch

Ref: constructive-io/constructive-planning#1062

Link to Devin session: https://app.devin.ai/sessions/815b2f2fc57441b0ae3430add6e90a70
Requested by: @pyramation

- Add SecretsLoader to @constructive-io/module-loader that queries
  config_secrets_module from metaschema and resolves secrets by name
  within a namespace scope (TTL-cached, multi-database aware)

- Extend FunctionContext with namespace, buckets, and apis fields
  for resource association (secrets go into env, not a new field)

- Add resolveAndInjectSecrets to ComputeWorker (compute-worker):
  reads required_secrets/required_configs from the function definition,
  resolves values from DB, injects into process.env before dispatch

- Add secret resolution to legacy Worker (job/worker) for backward
  compatibility with the new secret resolution flow

- Create stub function handlers:
  - namespace-provision: validates namespace_name, inserts created event
  - function-provision: validates service_url, checks required secrets
  - function-sync-resources: resolves all declared resources, logs warnings

Ref: constructive-io/constructive-planning#1062
@devin-ai-integration

Copy link
Copy Markdown

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

…date-only

- Remove fake InsertNamespaceEvent mutation from namespace-provision
- Simplify all three stubs to validate + log + return (no DB calls)
- Change ComputeWorker from resolveAndInjectSecrets → validateSecrets:
  checks existence only, does not inject into process.env
- Remove unused SecretsLoader from legacy worker (no function defs)
…lution

- Use metaschema.schema_and_table() for correct schema/table name resolution
- Replace fabricated secrets_table_name/decrypted_value with real columns
- Add two-step namespace resolution: name → UUID via namespace_module
- Query secrets by namespace_id (UUID FK), not namespace_name (doesn't exist)
- Decrypt secrets via pgp_sym_decrypt(value, key_id::text)
- Add namespace_name to PlatformFunctionDefinition type and discovery COLUMNS
- Use fn.namespace_name instead of fn.namespace_id in validateSecrets
@pyramation pyramation closed this Jun 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant