Skip to content

managed-services-redis.md

Latest commit

 

History

History
43 lines (29 loc) · 9.58 KB

managed-services-redis.md

File metadata and controls

43 lines (29 loc) · 9.58 KB
Raw

Managed services Tier 1 grows Valkey, with redis as a compatibility alias (#159)

  • Status: done
  • Date: 2026-06-13
  • Specs touched: SERVICE_PROVISIONING.md (# Catalog, # Implementation status, # Per-app isolation in shared instances), NEXT.md (# Managed-service lifecycle gaps, # Managed-service per-app key isolation), APP_MANIFEST.md (service types), DECISIONS.md 2026-06-13 (both entries), docs/dev/catalog-import-gaps.md (managed-redis — postiz → implemented)

Closes #159. The manifest schema already accepted services: {cache: {type: redis, version: "7"}}, but the brain provisioned only the SQL families (provisionedKinds = {postgres, mysql, mariadb}), so a redis declaration passed manifest check and then failed at install — a check/install asymmetry the catalog-import ledger kept hitting (managed-redis — postiz: Postiz needs Redis for BullMQ queues). This slice provisions the Redis-compatible substrate the same way Postgres/MySQL are, closing the asymmetry. Follows the MySQL-family slice (docs/progress/managed-services-mysql.md).

What was decided

The engine is Valkey, not Redis (DECISIONS.md 2026-06-13). The redis:7 image tag now resolves to Redis 7.4+ (RSALv2 + SSPLv1) and Redis 8+ is AGPLv3 — both license tracks are on malmo's avoid-list. So malmo runs Valkey, the Linux Foundation BSD-3-Clause fork of Redis 7.2.4 (RESP/ACL-compatible), and never the upstream Redis image at any version. valkey is a first-class managed type (single version line: 8); redis is kept as a pure compatibility alias that always provisions Valkey underneath — redis: "7" normalizes to the engine identity valkey: "8", so a redis:7 app and a valkey:8 app coalesce onto the one shared malmo-svc-valkey-8 instance. The grant stores the engine identity (Kind: "valkey", Version: "8"), so everything downstream keys off valkey and never sees "redis". Normalization happens once, early (normalizeEngine), so the lifecycle maps are valkey-only — there is deliberately no redis key, so no dead code path could pull the upstream image. The injected DSN keeps the universal redis:// scheme every client understands.

The per-app isolation model — an open question in NEXT.md — is resolved (DECISIONS.md 2026-06-13): a per-app ACL user with full keyspace, not a logical-DB-number split. Valkey has no database to scope a role to, so the per-app unit is an ACL user (ACL SETUSER <app> on >pw ~* &* +@all -@admin -flushall -flushdb -swapdb): every app authenticates as its own revocable credential and unauthenticated access is refused (a real auth boundary the DB-number split lacks), while the keyspace stays shared. -@admin keeps a compromised app off the ACL system / CONFIG / SHUTDOWN / replication, so it can't subvert the shared instance or the control plane, and -flushall -flushdb -swapdb keeps it from wiping the shared keyspace every other app reads from (subtracted by name rather than the blunt -@dangerous, which would also strip INFO/KEYS/SORT ordinary clients call on connect). The credential — not a key namespace — is the boundary; the shared keyspace still lets one app read another's keys, so per-app key confidentiality is the remaining deferred hardening (NEXT.md # Managed-service per-app key isolation), its clean form the isolated: true dedicated-instance flag.

What was done

internal/manifest

  • serviceVersions allowlists valkey (8) and keeps redis (7) for ecosystem compatibility; the unknown-type error message lists both.

internal/lifecycle

  • normalizeEngine(kind, version) — maps redis → valkey/8, everything else identity. Called at the top of the provisionServices per-dep loop and in serviceNetworkNames (so a redis-7 app attaches to the valkey-8 network), before anything touches the maps. Two real consumers, so it's not premature abstraction.
  • Maps are valkey-only: servicePort["valkey"] = 6379, serviceImageRepo["valkey"] = valkey/valkey (the BSD-3 image), serviceDSNScheme["valkey"] = redis (the universal RESP scheme), provisionedKinds["valkey"]. No redis key — redis is normalized away before any map lookup.
  • valkeyServiceCompose(version) — same shape as the SQL service composes (external --internal network, versioned valkey-8.malmo.internal DNS alias, fixed malmo-svc-valkey-8 exec handle, restart: unless-stopped). Runs valkey-server --aclfile /data/users.acl so per-app ACL users persist across a restart (ACLs are server config, not keyspace — they aren't in the RDB/AOF). REDISCLI_AUTH (valkey-cli honors the redis env var name) carries the superuser password into the container env so the brain's exec'd valkey-cli and the valkey-cli ping healthcheck authenticate without it ever reaching argv. (valkey/valkey:8 also ships redis-cli/redis-server as symlinks, but the code uses the honest valkey-* names.)
  • writeServiceDir bootstraps data/users.acl with the default (superuser) account (user default on >pw ~* &* +@all) so valkey can start; the entrypoint chowns the data dir to the valkey user, so it can rewrite the file on ACL SAVE. The superuser env var is VALKEY_SUPERUSER_PASSWORD (matches the compose ${...}).
  • provisionValkeyACL — ACL SETUSER the per-app user then ACL SAVE (persist to the aclfile), both via docker exec valkey-cli. The superuser auth rides REDISCLI_AUTH from the container env; the per-app password rides argv as the ACL >password token (base64url, no shell hazards), the same place the SQL families already carry per-app passwords — only the superuser password is kept out of argv.
  • dropServiceGrants generalized to a per-kind command list (one docker exec per command); Valkey runs ACL DELUSER + ACL SAVE. Best-effort, unchanged for the SQL engines.
  • provisionServices persists a Valkey grant with an empty DBName (the ACL user is the boundary; SQL grants keep db == role). serviceReadyProbe gains a valkey-cli ping | grep -q PONG branch.
  • writeEnv (internal/lifecycle/lifecycle.go) assembles the DSN without the /dbname suffix when the grant has no database, so Valkey injects redis://user:pw@valkey-8.malmo.internal:6379 (clients default to logical DB 0); Postgres/MySQL DSNs are unchanged. No code change beyond the maps — it keys entirely off the grant.

Tests (internal/lifecycle)

  • Hermetic (lifecycle_services_test.go, fake docker): TestInstallProvisionsValkeyViaRedisAlias (declared type: redis, "7", asserts a valkey/8 grant + malmo-svc-valkey-8 instance, no upstream redis-7 instance), TestInstallProvisionsValkeyNative (native type: valkey, "8"), TestRedisAndValkeyCoalesce (a redis-7 app and a valkey-8 app share one instance — no second ServiceUp, exactly one valkey-8 row — proving the alias), TestUninstallDropsValkeyACL (ACL DELUSER), and TestRedisProvisioningExecFailures (forces the SETUSER / SAVE execs to fail → install errors + clean rollback). Both install tests share assertValkeyProvisioned: ACL SETUSER … +@all -@admin -flushall -flushdb -swapdb + ACL SAVE issued; injected family has the valkey-8 host/6379/empty-NAME and a redis://…:6379 DSN with no db path; app attached to the svc network.
  • Live (dockerlive_test.go, real valkey/valkey:8): TestLiveRedisProvisioning declares type: redis, "7" to prove the alias end-to-end — lazy spinup of malmo-svc-valkey-8 with the external aclfile, the grant is valkey/8, the per-app ACL user is present in ACL LIST, authenticates with the injected password and reads/writes the keyspace (SET/GET via the redis:// URL form), and is gone from ACL LIST after ACL DELUSER on uninstall.
  • make check green.

What's next

  • Postiz (#128) is still blocked on its image-internal-path gaps (nonroot-data-ownership — postiz), not on Redis — its type: redis dependency is now first-class (served by Valkey) for any redis/valkey-needing app, but the app itself waits on userns-remap or a non-root upstream image.
  • The deferred Tier-1 pieces remain (NEXT.md # Managed-service lifecycle gaps): grace-shutdown after the last consumer uninstalls (services stay running today, Valkey included), cross-version migration, and at-rest encryption of the stored superuser + per-app passwords (plaintext today — the Valkey superuser password and per-app ACL passwords fold into the same NEXT.md # App-secret injection hardening gap).
  • Per-app Valkey key confidentiality is deferred (NEXT.md # Managed-service per-app key isolation). This slice subtracts the keyspace-destruction commands (-flushall -flushdb -swapdb) so no app can wipe the shared keyspace, but the keyspace is still shared, so an app can read another app's keys via GET/SCAN. Command ACLs can't close that (GET/SCAN are core); the clean fix is the isolated: true dedicated-instance flag (SERVICE_PROVISIONING.md # Per-app isolation). The same shared-keyspace constraint means uninstall drops the credential but cannot reclaim the app's keys — there is no per-app key namespace to flush, so an uninstalled app's keys linger until the instance is wiped; a dedicated instance (the isolated: true path) is also what makes uninstall reclaim the data.
  • Uninstall ACL SAVE is best-effort. dropServiceGrants runs ACL DELUSER then ACL SAVE; if the SAVE fails, a restart resurrects the (now grant-less, password-unknown) user from the unchanged aclfile. Harmless orphan, symmetric with the provisioning-side window, pruned by the reconcile loop that will own grace-shutdown (NEXT.md # Managed-service lifecycle gaps).