Inject shard-specific secrets into app compose templates (#138)

[ClaydeCode]

Closes #138.

What

Apps can now declare and receive shard-specific secrets in their docker-compose.yml.template:

services:
  db:
    environment:
      POSTGRES_PASSWORD: "{{ secret('db_password') }}"

On render, secret('<name>') returns an app-scoped secret — generating one (32 chars, [A-Za-z0-9], via the stdlib secrets CSPRNG) on first reference and persisting it in a new app_secrets table. Names are arbitrary and declared purely by referencing them. A secret is stable across re-renders, upgrades, and restarts (reused, never rotated), kept on uninstall (so a reinstall reuses it, matching the app's retained user_data), and not encrypted at rest — matching the decisions in the issue thread.

Templating syntax decision: the issue floated {{ secret.my-secret }}; a Jinja attribute access breaks on hyphens (parses as subtraction), so this uses the callable {{ secret('name') }} form — the alternative the issue also blessed ("identifiers cannot be mistaken for fixed keys").

The secret rides into the existing DB-snapshot backup automatically (db_snapshot.py discovers all non-yoyo tables), so it survives a restore.

How it works

render_docker_compose_template (app_installation/util.py) loads the app's existing secrets, renders with a SecretResolver that mints missing secrets synchronously during the single Jinja pass (Jinja is sync, the DB is not), persists new secrets before writing the compose file (a persistence failure aborts the render rather than leaving a compose file with secrets that were never stored), then writes the file.

Recommended reading order

1. migrations/shard-core-0002-app-secrets.sql — the app_secrets table (composite PK (app_name, name), no FK to installed_apps so secrets survive uninstall)
2. shard_core/database/app_secrets.py — get_all_for_app / idempotent insert
3. shard_core/service/app_installation/app_secrets.py — generate_secret, SecretResolver, load/persist
4. shard_core/service/app_installation/util.py — wiring into the render path
5. tests/test_app_secrets.py, tests/test_db_snapshot.py — coverage
6. agents.md — documents the template context + honest limitations

Verification

• tests/test_app_secrets.py (10 tests) and tests/test_db_snapshot.py (3) green.
• tests/test_app_installation.py (12, real Docker) green — confirms the change is safe on the real install path.
• ruff check clean; full suite collects (236 tests, no import breakage).

Review panel

Five reviewers ran (adversarial + test-adversary + DevEx + security + DB/migration), each given only the diff and the issue, instructed to refute. No blocking findings from any reviewer. Cheap advisories addressed:

#	Reviewer	Advisory	Resolution
1	DevEx (×2)	Service module secrets.py shadowed stdlib secrets (forced a pysecrets alias) and mismatched the DB module app_secrets.py	Fixed — renamed to app_secrets.py, dropped the alias (commit 5480bed)
2	Test adversary	test_distinct_names asserted rendered values but never that both secrets minted in one pass were persisted (a persist-loop bug storing only the first would slip through)	Fixed — added DB assertion (commit ced8b0d)
3	Test adversary	Reinstall-reuse (the headline lifecycle promise) only covered transitively	Fixed — added test_secret_reused_after_reinstall (commit ced8b0d)
4	Test adversary	generate_secret() never unit-tested directly (charset check was probabilistic)	Fixed — added length/charset/uniqueness unit test (commit ced8b0d)
—	Me (self-review)	agents.md claimed a "tracked separately" SandboxedEnvironment fix — no such issue is filed	Fixed — softened wording, no fabricated reference (commit 277fe75)

Advisories deliberately not taken, with reasons:

• Non-sandboxed Jinja lets a malicious app template escape and read any app's secrets (security + adversarial). This is pre-existing — portal/fs already render through the same non-sandboxed jinja2.Template, and custom-app upload is app-reachable per the architecture contract. This diff does not widen it: SecretResolver is built with only the current app's rows, so secret('x') can never name another app's secret without the pre-existing escape. Honestly disclosed in agents.md; a real SandboxedEnvironment fix is a separate, broader change.
• Secrets accumulate for permanently-removed apps (no purge path) — intended per the issue ("keep on uninstall"), symmetric with retained user_data, and rows are tiny.
• Read-generate-persist is non-atomic (TOCTOU under concurrent same-app renders) — unreachable today: renders are serialized through the single InstallationWorker, and boot re-render runs sequentially before the worker starts.
• Unbounded secret minting — dominated by the fact that a hostile template already has RCE; not worth a cap now.
• Worker end-to-end secret test — the direct-render tests are a fair proxy and the passing real-Docker test_app_installation.py already exercises the render path; skipped as costly for low marginal coverage.

🤖 Generated with Claude Code