Skip to content

Commit daf8dce

Browse files
h4x3rotabclaude
andcommitted
Reorganize repo layout and pin provider 0.3.0-beta.4
- Move sources into components/ (sidecar plumbing), workloads/ (patroni, webdemo), clusters/ (patroni-demo, new starter), support/, and docs/, from the flat top-level layout. - Add clusters/starter: minimal API+worker peer-VIP/service-VIP demo. - Pin phala-network/phala 0.3.0-beta.4 (official-Go-SDK build) in both clusters; refresh .terraform.lock.hcl. - Update publish.yml path filters for the new components/workloads directory layout. Verified on real cloud (patroni-demo, 8 CVMs): in-place env/compose updates with no slot replacement, mesh data path (webdemo LB via Consul+Envoy), and Patroni HA leader/replica. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 27743b5 commit daf8dce

58 files changed

Lines changed: 1332 additions & 162 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.dockerignore

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,16 @@
33
.local
44
snapshots
55

6-
cluster-example/.terraform
7-
cluster-example/terraform.tfstate*
8-
cluster-example/terraform.tfvars
9-
cluster-example/*.backup
10-
cluster-example/*.bak
6+
clusters/*/.terraform
7+
clusters/*/terraform.tfstate*
8+
clusters/*/terraform.tfvars
9+
clusters/*/*.backup
10+
clusters/*/*.bak
1111

12-
admission-broker/admission-broker
13-
admission-client/admission-client
14-
bootstrap-secrets/bootstrap-secrets
15-
mesh-conn/mesh-conn
16-
signaling/signaling
17-
signaling/icetest
18-
webdemo/webdemo
12+
components/admission-broker/admission-broker
13+
components/admission-client/admission-client
14+
components/bootstrap-secrets/bootstrap-secrets
15+
components/mesh-conn/mesh-conn
16+
components/signaling/signaling
17+
components/signaling/icetest
18+
workloads/webdemo/webdemo

.github/workflows/publish.yml

Lines changed: 24 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,8 @@ name: Publish service-mesh images
1515
#
1616
# `mesh-sidecar` is the consolidated platform-plumbing image. Its build
1717
# context is the repository root so its Dockerfile
18-
# can pull the Go sources from sibling subdirs. The other three images
19-
# build from their own subdirs.
18+
# can pull Go sources from component subdirs. The other three images
19+
# build from their workload/component subdirs.
2020
#
2121
# Verifying a published image (consumer side):
2222
#
@@ -28,25 +28,25 @@ on:
2828
push:
2929
branches: [main]
3030
paths:
31-
- 'bootstrap-secrets/**'
32-
- 'admission-broker/**'
33-
- 'admission-client/**'
34-
- 'mesh-conn/**'
35-
- 'mesh-sidecar/**'
36-
- 'patroni/**'
37-
- 'webdemo/**'
38-
- 'signaling/**'
31+
- 'components/bootstrap-secrets/**'
32+
- 'components/admission-broker/**'
33+
- 'components/admission-client/**'
34+
- 'components/mesh-conn/**'
35+
- 'components/mesh-sidecar/**'
36+
- 'components/signaling/**'
37+
- 'workloads/patroni/**'
38+
- 'workloads/webdemo/**'
3939
- '.github/workflows/publish.yml'
4040
pull_request:
4141
paths:
42-
- 'bootstrap-secrets/**'
43-
- 'admission-broker/**'
44-
- 'admission-client/**'
45-
- 'mesh-conn/**'
46-
- 'mesh-sidecar/**'
47-
- 'patroni/**'
48-
- 'webdemo/**'
49-
- 'signaling/**'
42+
- 'components/bootstrap-secrets/**'
43+
- 'components/admission-broker/**'
44+
- 'components/admission-client/**'
45+
- 'components/mesh-conn/**'
46+
- 'components/mesh-sidecar/**'
47+
- 'components/signaling/**'
48+
- 'workloads/patroni/**'
49+
- 'workloads/webdemo/**'
5050
- '.github/workflows/publish.yml'
5151
workflow_dispatch:
5252

@@ -69,18 +69,16 @@ jobs:
6969
matrix:
7070
include:
7171
# `mesh-sidecar` builds with the parent dir as context so
72-
# its Dockerfile can pull bootstrap-secrets/, mesh-conn/,
73-
# admission-broker/, and admission-client/ Go sources from
74-
# siblings.
72+
# its Dockerfile can pull Go sources from components/.
7573
- name: mesh-sidecar
7674
context: .
77-
dockerfile: mesh-sidecar/Dockerfile
75+
dockerfile: components/mesh-sidecar/Dockerfile
7876
- name: patroni
79-
context: ./patroni
77+
context: ./workloads/patroni
8078
- name: webdemo
81-
context: ./webdemo
79+
context: ./workloads/webdemo
8280
- name: signaling
83-
context: ./signaling
81+
context: ./components/signaling
8482

8583
steps:
8684
- uses: actions/checkout@v4
@@ -115,7 +113,7 @@ jobs:
115113
context: ${{ matrix.context }}
116114
# Most images use the default Dockerfile in the context.
117115
# `mesh-sidecar` overrides this to point at
118-
# mesh-sidecar/Dockerfile while keeping the parent context.
116+
# components/mesh-sidecar/Dockerfile while keeping the parent context.
119117
file: ${{ matrix.dockerfile || format('{0}/Dockerfile', matrix.context) }}
120118
platforms: linux/amd64
121119
push: ${{ github.event_name != 'pull_request' }}

.gitignore

Lines changed: 15 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -8,17 +8,20 @@ snapshots/
88
.env
99

1010
# Compiled Go binaries
11-
admission-broker/admission-broker
12-
admission-client/admission-client
13-
bootstrap-secrets/bootstrap-secrets
14-
mesh-conn/mesh-conn
15-
signaling/signaling
16-
signaling/icetest
17-
webdemo/webdemo
11+
components/admission-broker/admission-broker
12+
components/admission-client/admission-client
13+
components/bootstrap-secrets/bootstrap-secrets
14+
components/mesh-conn/mesh-conn
15+
components/signaling/signaling
16+
components/signaling/icetest
17+
workloads/webdemo/webdemo
1818

1919
# Terraform local state. Keep .terraform.lock.hcl tracked.
20-
cluster-example/.terraform/
21-
cluster-example/terraform.tfstate*
22-
cluster-example/terraform.tfvars
23-
cluster-example/*.backup
24-
cluster-example/*.bak
20+
clusters/*/.terraform/
21+
clusters/*/terraform.tfstate*
22+
clusters/*/terraform.tfvars
23+
clusters/*/*.backup
24+
clusters/*/*.bak
25+
26+
# Legacy local state after moving the demo cluster to clusters/patroni-demo.
27+
cluster-example/

CONTRIBUTING.md

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -16,30 +16,32 @@ Install:
1616
Validate the Terraform example:
1717

1818
```bash
19-
terraform -chdir=cluster-example init -upgrade
20-
terraform -chdir=cluster-example validate
19+
terraform -chdir=clusters/patroni-demo init -upgrade
20+
terraform -chdir=clusters/patroni-demo validate
21+
terraform -chdir=clusters/starter init -upgrade
22+
terraform -chdir=clusters/starter validate
2123
```
2224

2325
Run local Go tests:
2426

2527
```bash
26-
(cd admission-broker && go test ./...)
27-
(cd signaling && go test ./...)
28-
(cd webdemo && go test ./...)
28+
(cd components/admission-broker && go test ./...)
29+
(cd components/signaling && go test ./...)
30+
(cd workloads/webdemo && go test ./...)
2931
```
3032

3133
Run Go 1.24 module tests through Docker when needed:
3234

3335
```bash
34-
docker run --rm -v "$PWD:/src" -w /src/admission-client golang:1.24 go test ./...
35-
docker run --rm -v "$PWD:/src" -w /src/bootstrap-secrets golang:1.24 go test ./...
36-
docker run --rm -v "$PWD:/src" -w /src/mesh-conn golang:1.24 go test ./...
36+
docker run --rm -v "$PWD:/src" -w /src/components/admission-client golang:1.24 go test ./...
37+
docker run --rm -v "$PWD:/src" -w /src/components/bootstrap-secrets golang:1.24 go test ./...
38+
docker run --rm -v "$PWD:/src" -w /src/components/mesh-conn golang:1.24 go test ./...
3739
```
3840

3941
Build the main sidecar image:
4042

4143
```bash
42-
docker build -t service-mesh-mesh-sidecar-check -f mesh-sidecar/Dockerfile .
44+
docker build -t service-mesh-mesh-sidecar-check -f components/mesh-sidecar/Dockerfile .
4345
```
4446

4547
## Pull Request Expectations

README.md

Lines changed: 57 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ The default deployment is a public web backend plus a private HA
2222
PostgreSQL service. Every workload CVM must pass attestation admission
2323
before it receives Consul and Envoy credentials.
2424

25-
![Default demo architecture](assets/demo-architecture.svg)
25+
![Default demo architecture](docs/assets/demo-architecture.svg)
2626

2727
Applications use stable names such as `webdemo`, `postgres-master`, and
2828
`postgres-replica`; they do not need to know which CVM is currently
@@ -38,7 +38,7 @@ You need:
3838

3939
```bash
4040
git clone https://github.com/Dstack-TEE/service-mesh.git
41-
cd service-mesh/cluster-example
41+
cd service-mesh/clusters/patroni-demo
4242
cp terraform.tfvars.example terraform.tfvars
4343
export PHALA_CLOUD_API_KEY=phak_...
4444
terraform init -upgrade
@@ -106,7 +106,7 @@ mesh path. The database is private; it exposes `postgres-master` and
106106

107107
## Demo Inputs
108108

109-
`cluster-example/terraform.tfvars.example` contains the default demo
109+
`clusters/patroni-demo/terraform.tfvars.example` contains the default demo
110110
values. Edit the copied `terraform.tfvars` when you want to change
111111
replica counts, cluster name, bootstrap endpoint, or image tags.
112112

@@ -119,6 +119,39 @@ Public gateway URLs are derived from the Phala Terraform provider's
119119
`gateway_base_domain` attribute, so this template does not require a
120120
hardcoded gateway suffix.
121121

122+
## Starter Template
123+
124+
Use `clusters/starter` when you want the smallest custom mesh to copy:
125+
one public API workload group and one private worker workload group.
126+
It deliberately has no database and no Patroni role tags.
127+
128+
```bash
129+
cd service-mesh/clusters/starter
130+
cp terraform.tfvars.example terraform.tfvars
131+
export PHALA_CLOUD_API_KEY=phak_...
132+
terraform init -upgrade
133+
terraform apply
134+
135+
API=$(terraform output -raw api_url)
136+
curl "$API/hello"
137+
curl "$API/worker"
138+
```
139+
140+
`/hello` returns the API instance identity. `/worker` makes the API call
141+
`http://worker:9090/hello` through the private mesh.
142+
143+
## Repository Layout
144+
145+
- `clusters/patroni-demo`: full demo cluster, including its Terraform
146+
definition and compose files.
147+
- `clusters/starter`: minimal API + worker cluster template.
148+
- `components`: reusable platform components built into the sidecar or
149+
support images.
150+
- `workloads`: demo workload images used by the Patroni demo.
151+
- `support`: helper services that are useful during bootstrap or local
152+
operation but are not a cluster definition.
153+
- `docs`: architecture, operations, and design notes.
154+
122155
## Default Demo Topology
123156

124157
This section describes the shipped demo only: a public web backend plus
@@ -181,15 +214,15 @@ flowchart LR
181214
coord -. admits .-> api2
182215
```
183216

184-
Declare workload groups, producer backends, and callable routes in
185-
`cluster-example/cluster.tf`:
217+
The starter cluster in `clusters/starter/cluster.tf` is a small version
218+
of this shape:
186219

187220
```hcl
188221
locals {
189222
workload_groups = {
190223
api = {
191224
replicas = 2
192-
compose = "${path.module}/../compose/api.yaml"
225+
compose = "${path.module}/compose/api.yaml"
193226
images = { API_IMAGE = var.api_image }
194227
secrets = {}
195228
backends = {
@@ -207,7 +240,7 @@ locals {
207240
208241
worker = {
209242
replicas = 5
210-
compose = "${path.module}/../compose/job-worker.yaml"
243+
compose = "${path.module}/compose/worker.yaml"
211244
images = { WORKER_IMAGE = var.worker_image }
212245
secrets = {}
213246
backends = {
@@ -233,7 +266,7 @@ starts an egress Envoy on every workload CVM so calls work even when the
233266
caller does not host the target backend.
234267

235268
Run each application container in its own group's compose file beside
236-
the existing `sidecar` service. For example, `compose/api.yaml` would
269+
the existing `sidecar` service. For example, `clusters/starter/compose/api.yaml` would
237270
contain:
238271

239272
```yaml
@@ -251,7 +284,7 @@ services:
251284
condition: service_healthy
252285
```
253286
254-
`compose/job-worker.yaml` would have the same sidecar and a `worker`
287+
`clusters/starter/compose/worker.yaml` would have the same sidecar and a `worker`
255288
service that binds `127.0.0.1:9090`.
256289

257290
After deployment, code in either workload group can call:
@@ -274,7 +307,7 @@ For a backend port listed in `workload_groups`, avoid binding `0.0.0.0:<port>`.
274307
That also captures the local mesh address for the same port. Bind
275308
`127.0.0.1:<port>` for mesh traffic; if the same service is public
276309
through the Phala gateway, also bind the host interface address. The
277-
demo [`webdemo`](webdemo/main.go) shows that dual-bind pattern.
310+
demo [`webdemo`](workloads/webdemo/main.go) shows that dual-bind pattern.
278311

279312
## Service Definition Reference
280313

@@ -335,7 +368,7 @@ identity `spiffe://<cluster>/api`.
335368
The shipped backend-plus-database demo uses
336369
`registration.owner = "workload"` for Postgres because runtime metadata
337370
belongs to Patroni. The sidecar still creates a Connect-aware stub so
338-
Envoy can start, but `patroni/entrypoint.sh` updates dynamic metadata
371+
Envoy can start, but `workloads/patroni/entrypoint.sh` updates dynamic metadata
339372
such as role tags and health checks. `postgres-master` and
340373
`postgres-replica` are two routes to one Patroni/Postgres backend:
341374

@@ -344,7 +377,7 @@ locals {
344377
workload_groups = {
345378
postgres = {
346379
replicas = var.postgres_replicas
347-
compose = "${path.module}/../compose/postgres.yaml"
380+
compose = "${path.module}/compose/postgres.yaml"
348381
backends = {
349382
postgres = {
350383
port = 5432
@@ -367,7 +400,7 @@ locals {
367400

368401
Both routes share `backend = "postgres"` and select the same parent
369402
service, `var.cluster_name`. Patroni uses Consul for leader election,
370-
and `patroni/entrypoint.sh` runs the role-watcher that publishes the
403+
and `workloads/patroni/entrypoint.sh` runs the role-watcher that publishes the
371404
current `master` or `replica` tag. The Consul resolver for each route
372405
selects the matching tag, so clients keep dialing
373406
`postgres-master:5432` and `postgres-replica:5432` across failovers.
@@ -416,31 +449,33 @@ design.
416449
Validate Terraform:
417450

418451
```bash
419-
terraform -chdir=cluster-example init -upgrade
420-
terraform -chdir=cluster-example validate
452+
terraform -chdir=clusters/patroni-demo init -upgrade
453+
terraform -chdir=clusters/patroni-demo validate
454+
terraform -chdir=clusters/starter init -upgrade
455+
terraform -chdir=clusters/starter validate
421456
```
422457

423458
Run local Go tests:
424459

425460
```bash
426-
(cd admission-broker && go test ./...)
427-
(cd signaling && go test ./...)
428-
(cd webdemo && go test ./...)
461+
(cd components/admission-broker && go test ./...)
462+
(cd components/signaling && go test ./...)
463+
(cd workloads/webdemo && go test ./...)
429464
```
430465

431466
Some modules use Go 1.24. Run them through Docker if your host Go is
432467
older:
433468

434469
```bash
435-
docker run --rm -v "$PWD:/src" -w /src/admission-client golang:1.24 go test ./...
436-
docker run --rm -v "$PWD:/src" -w /src/bootstrap-secrets golang:1.24 go test ./...
437-
docker run --rm -v "$PWD:/src" -w /src/mesh-conn golang:1.24 go test ./...
470+
docker run --rm -v "$PWD:/src" -w /src/components/admission-client golang:1.24 go test ./...
471+
docker run --rm -v "$PWD:/src" -w /src/components/bootstrap-secrets golang:1.24 go test ./...
472+
docker run --rm -v "$PWD:/src" -w /src/components/mesh-conn golang:1.24 go test ./...
438473
```
439474

440475
Build the sidecar image:
441476

442477
```bash
443-
docker build -t service-mesh-mesh-sidecar-check -f mesh-sidecar/Dockerfile .
478+
docker build -t service-mesh-mesh-sidecar-check -f components/mesh-sidecar/Dockerfile .
444479
```
445480

446481
## More Docs

admission-broker/go.mod

Lines changed: 0 additions & 3 deletions
This file was deleted.

0 commit comments

Comments
 (0)