Skip to content

Commit 66e8565

Browse files
h4x3rotabh4x3rotab
committed
Track Terraform slot API handoff
1 parent 0e12934 commit 66e8565

3 files changed

Lines changed: 183 additions & 9 deletions

File tree

‎consul-postgres-ha/PROGRESS.md‎

Lines changed: 12 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -33,28 +33,31 @@ deployment model, trust model, or operational status.
3333

3434
## Open Work
3535

36+
- Terraform slot API: integrate the newly implemented Phala Cloud
37+
slot API into the Terraform deployment flow. Tracking note:
38+
`design/terraform-slot-api.md`.
3639
- KMS policy: pin the long-term KMS public key or equivalent KMS
3740
identity in Terraform and require the verifier-proven KMS evidence
3841
to match it.
3942
- Production OS policy: when a production profile is enabled, include
4043
the expected OS image hash in both Phala launch config and admission
4144
policy.
42-
- Peer identity hardening: keep `peer_id` as the Terraform-declared
43-
Consul node namespace, but bind it to a platform-stable instance
44-
identifier when Phala exposes one without creating a deployment
45-
cycle.
45+
- Secret lifecycle: move remaining cluster-wide-identical secrets
46+
out of Terraform env into TEE-rooted generation/distribution.
47+
- Long soak: run the real cluster for at least one hour with zero ACL
48+
authorization failures, zero attestation rejections, and stable
49+
Patroni leader/replica health.
4650
- Token lifecycle: keep non-expiring tokens for the first integrated
4751
version; next step is one active token per attested workload
4852
instance, then bounded TTL plus renewal once Consul, Envoy, and
4953
Patroni reload paths are explicit.
5054
- Upgrade policy: replace static compose-hash allowlists with a
5155
policy server or on-chain bridge that admits new revisions under
5256
signed policy epochs.
53-
- Secret lifecycle: move remaining cluster-wide-identical secrets
54-
out of Terraform env into TEE-rooted generation/distribution.
55-
- Long soak: run the real cluster for at least one hour with zero ACL
56-
authorization failures, zero attestation rejections, and stable
57-
Patroni leader/replica health.
57+
- Peer identity hardening: keep `peer_id` as the Terraform-declared
58+
Consul node namespace, but bind it to a platform-stable instance
59+
identifier when Phala exposes one without creating a deployment
60+
cycle.
5861

5962
## Decisions
6063

‎consul-postgres-ha/design/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ in their face.
1313
| Doc | What |
1414
|---|---|
1515
| [`attestation-admission.md`](attestation-admission.md) | Use dstack TEE attestation as the mesh-conn admission credential, replacing/augmenting the shared TURN HMAC. Phased plan: per-app-id first, Consul-KV-rooted policy later. |
16+
| [`terraform-slot-api.md`](terraform-slot-api.md) | Integrate the Phala Cloud slot API into Terraform so mesh members have stable logical identities across CVM replacement and future attestation policy binding. |
1617

1718
Each doc includes:
1819

‎consul-postgres-ha/design/terraform-slot-api.md‎

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
# Design: Terraform Slot API Integration
2+
3+
## Status
4+
5+
Handoff in progress.
6+
7+
- Paseo agent: `5219012b-97d5-4d2c-bcbb-a24120e0ac30`
8+
- Agent worktree:
9+
`/home/h4x/.paseo/worktrees/1izuidyu/brawny-seahorse`
10+
- Target repo for provider work: `Phala-Network/phala-cloud`
11+
- Local consumer repo: `consul-postgres-ha`
12+
13+
This note exists so we can recover the slot-management context even if
14+
the Paseo logs are not immediately available.
15+
16+
## Why This Matters
17+
18+
The service mesh needs stable logical members such as `coordinator-1`
19+
and `worker-4`. Today, a concrete CVM can be replaced during upgrades,
20+
failures, or re-provisioning, and the platform-level VM identifier can
21+
change. The mesh, Consul node namespace, Patroni topology, and
22+
attestation policy need a stable Terraform-declared member identity
23+
that survives that replacement.
24+
25+
The Phala Cloud slot API is the platform primitive for this:
26+
27+
- `app_id` identifies the application or replica set.
28+
- `slot` identifies a stable logical member inside the app.
29+
- `vm_uuid` identifies the current concrete CVM occupying the slot.
30+
- `instance_id` identifies the runtime, network, or workload instance
31+
where applicable.
32+
33+
For this repo, the slot should become the bridge between Terraform
34+
intent and the runtime member. Compose hash remains revision evidence;
35+
it should not be the workload key.
36+
37+
## Source Context
38+
39+
The original feature request is:
40+
41+
- https://github.com/Phala-Network/phala-cloud/issues/243
42+
43+
The issue describes the desired Terraform direction:
44+
45+
- model `phala_app` as the replica set or application;
46+
- model `phala_app_instance` as a stable logical member;
47+
- let replacement create a new concrete CVM while preserving the slot;
48+
- keep slot names user chosen, immutable after creation, and unique
49+
under one app.
50+
51+
The issue also sketched these API operations:
52+
53+
- `GET /apps/{app_id}/instances`
54+
- `GET /apps/{app_id}/instances/{slot}`
55+
- `POST /apps/{app_id}/instances`
56+
- `PATCH /apps/{app_id}/instances/{slot}`
57+
- `DELETE /apps/{app_id}/instances/{slot}`
58+
- `POST /apps/{app_id}/instances/{slot}/replace`
59+
60+
The implementing agent must verify the actual just-implemented API in
61+
the current `phala-cloud` tree instead of assuming the issue text is
62+
still exact.
63+
64+
## Desired Provider Behavior
65+
66+
The provider should expose stable slot management in a way Terraform can
67+
reason about directly.
68+
69+
Preferred shape, if it matches the platform API cleanly:
70+
71+
```hcl
72+
resource "phala_app" "mesh" {
73+
# app-level definition
74+
}
75+
76+
resource "phala_app_instance" "worker_1" {
77+
app_id = phala_app.mesh.id
78+
slot = "worker-1"
79+
80+
# instance launch/update fields, if these belong at slot scope
81+
}
82+
```
83+
84+
An alternate shape may be acceptable if the platform API strongly
85+
pushes instance management under `phala_app`, but the result must still
86+
let Terraform address one stable logical member without depending on the
87+
current `vm_uuid`.
88+
89+
Provider semantics to preserve:
90+
91+
- `slot` is stable identity and should force replacement if changed.
92+
- Read must distinguish the stable slot from the current concrete CVM.
93+
- Replacement should intentionally move the slot to a new CVM without
94+
losing the logical member identity.
95+
- Existing public provider behavior must not regress.
96+
- Import should be possible using an address that includes `app_id` and
97+
`slot`, if the API supports it.
98+
99+
## Service Mesh Integration Plan
100+
101+
Once provider support lands and is released or locally pinned for
102+
testing, update this repo so the real cluster uses slots explicitly.
103+
104+
Expected mapping:
105+
106+
- `coordinator-1`, `coordinator-2`, `coordinator-3` are stable slots.
107+
- `worker-1`, `worker-2`, `worker-3` are stable slots.
108+
- The Consul node name and mesh `peer_id` should be derived from the
109+
Terraform slot, not from a generated VM identifier.
110+
- Admission policy should continue to verify compose hash as revision
111+
evidence and should later bind the quoted identity to the
112+
platform-stable slot or instance evidence exposed by Phala Cloud.
113+
114+
This is a prerequisite for deterministic rolling replacement and for a
115+
cleaner future upgrade policy. It also avoids treating compose hash as a
116+
workload key, since multiple logical workloads may legitimately share
117+
the same compose hash.
118+
119+
## Agent Handoff
120+
121+
The following task was sent to agent
122+
`5219012b-97d5-4d2c-bcbb-a24120e0ac30`:
123+
124+
- discover the current slot API implementation in `phala-cloud`;
125+
- implement Terraform provider support for stable slot management;
126+
- keep the work scoped to slot management, not KMS, OS, or attestation;
127+
- do not revert unrelated changes in its worktree;
128+
- add focused tests for read/create/update/delete/replace/import where
129+
the API makes those behaviors meaningful;
130+
- report changed files, tests run, and the recommended Terraform shape.
131+
132+
Reason given to the agent: this repo needs stable logical members for
133+
HA mesh operation, deterministic rolling replacement, and future
134+
attestation binding to a Terraform-declared member rather than the
135+
current concrete CVM.
136+
137+
## Success Criteria
138+
139+
- Terraform provider can manage Phala app slots as first-class stable
140+
logical members or an equivalently addressable structure.
141+
- Provider state exposes both the stable slot and current concrete CVM
142+
identity when available.
143+
- Existing provider tests still pass.
144+
- New slot tests cover the intended lifecycle.
145+
- `consul-postgres-ha/cluster-example` can be updated without copying
146+
app or instance configuration between unrelated Terraform resources.
147+
- A real apply/destroy can create the 3-coordinator / 3-worker cluster
148+
using public or explicitly pinned provider artifacts.
149+
150+
## Open Questions
151+
152+
- Does the released API expose slot replacement as a separate operation
153+
or as an update/create behavior?
154+
- Which fields belong to `phala_app` versus an instance or slot
155+
resource?
156+
- Can import be stable on `app_id/slot`, or does the API require an
157+
internal ID?
158+
- Does the API expose enough stable evidence to bind future attestation
159+
admission to the slot directly?
160+
- What is the migration path from the current deployment state to
161+
slot-managed state without destroying healthy clusters unexpectedly?
162+
163+
## Next Local Steps After Agent Callback
164+
165+
- Review the provider patch and Terraform shape.
166+
- Update this repo's provider version or local override for testing.
167+
- Wire `cluster-example` coordinator and worker definitions to slots.
168+
- Run public-provider or pinned-provider real cluster apply/destroy.
169+
- Update `PROGRESS.md` with the tested status and delete this design
170+
note once the work lands.

0 commit comments

Comments
 (0)