Skip to content

progress.md

Latest commit

 

History

History
104 lines (95 loc) · 5.29 KB

progress.md

File metadata and controls

104 lines (95 loc) · 5.29 KB
Raw

Project Progress

This tracks the service-mesh project as a whole, not only attestation. Keep it current when landing changes that alter the deployment model, trust model, or operational status.

Current Status

  • Terraform deploys a 3-coordinator / separate webdemo and postgres workload-group real Phala Cloud cluster with public provider phala-network/phala v0.3.0-beta.3 using the named-slot model.
  • mesh-conn direct connectivity is enabled by default; relay-only configuration was removed.
  • Consul server quorum, Consul clients, Envoy Connect sidecars, and Patroni HA Postgres are running through the mesh.
  • Attestation admission gates Consul node and service token issuance through real dstack quotes and dstack-verifier; the cluster example enables it by default and generates a Consul management token when one is not supplied.
  • Workload-group preflight compose hashes are generated by Terraform and used as revision evidence in the admission policy.
  • Admission policy checks KMS key-provider evidence. When production_profile = true, it also checks verifier-proven OS image hash evidence against the Terraform preflight hash.

Recently Validated

  • Full real-cluster Terraform apply with preflight-backed admission policy.
  • Public Terraform provider path, with no local provider override: terraform init -upgrade, apply, no-op plan, and destroy against the public phala-network/phala 0.3 beta provider line.
  • Production-profile real-cluster apply on prod dstack-0.5.7, with all six CVMs reporting is_dev = false and OS image hash 761c05d282c81abeae2d1a8f6d5b1e039c8ce14cc95a6da020b9ed2ff1056816.
  • Real attestation path on all workers for node, webdemo, and postgres identities.
  • Negative admission test: a valid quote claiming the wrong peer_id is rejected by policy.
  • Worker sidecar restart: node token, webdemo token, and postgres token are re-issued through attestation before the worker rejoins.
  • Consul health after tests: zero critical checks, all coordinators and workers alive.
  • Broker unit tests cover KMS mismatch rejection and production OS image hash mismatch rejection.

Open Work

  • Rollout driver: clusters/patroni-demo/rollout.sh is paused until it is updated for the workload-group topology and the provider 0.3 app-revision fan-out path. Direct Terraform apply is the current deployment path; the script's Consul-health scaffolding can be reused for a future workload-aware drain-and-roll workflow.
  • Secret lifecycle: role-local secrets are now derived in-TEE by bootstrap-secrets (consumers share one app_id, so GetKey yields identical bytes) and never appear in tfstate — the Consul ACL management token (coordinators) and the Patroni superuser/replication passwords (postgres), both verified on real cloud. Remaining: only the gossip key, which is cross-role (coordinators AND workload apps need byte-identical bytes under different app_ids), so it needs TEE-rooted generation + attested distribution, not just derivation.
  • Long soak: run the real cluster for at least one hour with zero ACL authorization failures, zero attestation rejections, and stable Patroni leader/replica health.
  • Deep prod-profile operational validation: prod OS images disable SSH, so use Phala logs/API evidence for admission and container health. Run dev-image validation when direct consul/patronictl CLI access is required.
  • Token lifecycle: keep non-expiring tokens for the first integrated version; next step is one active token per attested workload instance, then bounded TTL plus renewal once Consul, Envoy, and Patroni reload paths are explicit.
  • Upgrade policy: replace static compose-hash allowlists with a policy server or on-chain bridge that admits new revisions under signed policy epochs.
  • Peer identity hardening: keep peer_id as the Terraform-declared Consul node namespace, but bind it to a platform-stable instance identifier when Phala exposes one without creating a deployment cycle.
  • Parallel deploys blocked on KMS nonce race: concurrent app creation (two terraform applys, or one apply with -parallelism > 1) fails with [ERR-02-009] ... unique constraint "ix_dstack_app_nonces_address" because the KMS obtain_app_id allocates the per-team-wallet nonce with an unlocked read-modify-write. Verified on provider 0.3.0-beta.4. Use -parallelism=1 and one apply at a time until the upstream fix lands (route obtain_app_id through the existing .with_for_update() path): Phala-Network/phala-cloud-monorepo#1544. The provider exposes custom_app_id + nonce for manual pre-allocation, but that can't cover two separate concurrent applies.

Decisions

  • Workload identity is the Terraform-declared identity; compose hash is revision evidence, not the workload key.
  • report_data is the 64-byte TDX REPORTDATA field: SHA-256(binding_statement_json || nonce) || 32 zero bytes.
  • The broker relies on dstack-verifier for quote and event-log semantics. It does not parse raw RTMR event-log internals itself.
  • peer_id is the logical mesh/Consul node label, for example worker-3. It is not a secret. It is quoted in report_data and matched against Terraform policy before node-scoped tokens are issued.
  • Production image policy should prefer digest-pinned Compose image references.