You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
fix(tee): pin SecretVM to v0.0.27 stable Production release (#733)
## Summary
Pins SecretVM artifacts to
**[v0.0.27](https://github.com/scrtlabs/secret-vm-build/releases/tag/v0.0.27)**
— the latest non-prerelease Production release on GitHub — replacing
`v0.0.27-alpha.1`.
| Artifact | SHA256 (verified against ISO download + registry) |
|----------|---------------------------------------------------|
| TDX `rootfs-prod-v0.0.27-tdx.iso` |
`1a5d9ea46607fe46f0ad8055ee01afa033aaded7251a38253baff470d33506c8` |
| SEV `rootfs-prod-v0.0.27-sev.iso` |
`17122d51dbaed224d08bc0c17e72d7047d60175c7274a0d366167fc29b3fe4cf` |
SEV registry has an exact `prod` / `v0.0.27` entry — no
`SECRETVM_SEV_REGISTRY_ARTIFACTS_VER` override needed.
**Portal note:** The SecretVM UI may still show `v0.0.26-beta.1` until
SCRT rolls the cluster. CI/manifests pin to v0.0.27 regardless.
Post-deploy RTMR3 verify on an existing test VM will only pass once that
VM runs v0.0.27 rootfs.
**Follow-up (do not forget):** After TEST validation, bring this same
pin to `dev`, then refresh #732 (TEST→MAIN) before merge.
**#732 is on hold** until this lands on TEST and a TEE build is
validated.
## Test plan
- [ ] Merge to `test` and trigger `build.yml` on `test` branch (TEE job)
- [ ] Confirm `Download SecretVM rootfs (TDX + SEV)` passes SHA256
verification
- [ ] Confirm `Compute SEV-SNP measurement (per template)` succeeds
(registry lookup for `v0.0.27`)
- [ ] Confirm attestation manifest includes `intel_tdx` + `amd_sev_snp`
blocks with `secretvm_release: v0.0.27`
- [ ] If `SECRETVM_*` secrets configured: note whether deploy+RTMR3
verify passes (may fail until portal offers v0.0.27 or test VM is
recreated)
Made with [Cursor](https://cursor.com)
> -**Phase 1b (RTMR3 computation + automated deployment + post-deploy attestation)** — DONE (v6.0.0); **SecretVM release pin updated to `v0.0.27-alpha.1`** (the only release the SecretVM portal currently accepts for *new* VM provisioning, 2026-04-29)
> -**Phase 1c (proxy-router code: consumer verifies P-Node)** — DONE (v6.0.0, refined through v6.2.x); SEV-SNP path verifies via the dedicated `quote-parse-sev` portal endpoint and a per-template golden table (PR #718)
10
10
> -**Phase 2a (P-Node verifies backend LLM: CPU quote, TLS pinning, RTMR3 replay for TDX *and* GCTX launch-digest replay for SEV, CPU-GPU binding, NRAS)** — DONE (v7.0.0; SEV branch added by PR #718, VMType fix in PR #720)
11
11
> -**CI/CD SEV measurement signing** — DONE (this PR — SEV launch digest computed for all 5 portal templates and attached to the cosign-signed manifest, alongside baked log-level fields in `baked_env`)
@@ -358,7 +358,7 @@ This is a **release cadence** concern, not a clock-based TTL. A version doesn't
358
358
- Can be run locally by providers/consumers for independent verification
- Pins SecretVM release version (currently **`v0.0.27-alpha.1`** — the only release the SecretVM portal accepts for *new* VM provisioning as of 2026-04-29) and rootfs variant base (`rootfs-prod`)
361
+
- Pins SecretVM release version (currently **`v0.0.27`** — GitHub latest stable Production release as of 2026-05-23) and rootfs variant base (`rootfs-prod`)
362
362
- Pins TDX **and** SEV rootfs URLs + their expected SHA-256 hashes
363
363
- Pins the SEV artifact registry URL (`SECRETVM_SEV_REGISTRY_URL`) and the registry vm_type (`SECRETVM_SEV_REGISTRY_VM_TYPE=prod`) — used by `compute-sev-measurement.py` to find the right entry
364
364
- All pipeline references to rootfs filenames are derived from `SECRETVM_ROOTFS_VARIANT` (no hardcoded filenames in `build.yml`); the platform suffix (`-tdx` / `-sev`) is appended at use site
@@ -383,7 +383,7 @@ This is a **release cadence** concern, not a clock-based TTL. A version doesn't
383
383
→ sign and attach manifest → upload deployed compose
- Inputs: SEV registry JSON entry (kernel/initrd/ovmf hashes, OVMF section table, sev_hashes_table_gpa, sev_es_reset_eip), the deployed `docker-compose.tee.yml`, and a vCPU template.
476
476
- Emits one `measurement` per template (small=1, medium=2, large=4, 2xlarge=8, 4xlarge=16).
477
-
- Parity-tested against the Go runtime via `internal/attestation/sev_python_parity_test.go` (uses `t.TempDir()` + the v0.0.27-alpha.1 fixture; the test runs the Python script as a subprocess and asserts identical hex output for all 5 templates).
477
+
- Parity-tested against the Go runtime via `internal/attestation/sev_python_parity_test.go` (uses `t.TempDir()` + the v0.0.27 fixture; the test runs the Python script as a subprocess and asserts identical hex output for all 5 templates).
**Implementation:**`.github/tee/secretvm.env` pins the release version, rootfs variant, and BOTH the TDX and SEV rootfs URLs + SHA-256s. The pipeline derives all artifact filenames, cache keys, and download paths from `SECRETVM_RELEASE` and `SECRETVM_ROOTFS_VARIANT` (with the platform suffix `-tdx`/`-sev` appended). The attestation manifest includes `measurements.intel_tdx.{secretvm_release, rootfs_variant, rootfs_sha256}` and `measurements.amd_sev_snp.{secretvm_release, rootfs_sha256, kernel_hash, initrd_hash, ovmf_hash, …}`.
492
492
493
-
**Current pin (2026-04-29):**`v0.0.27-alpha.1`. SCRT Labs flipped the SecretVM portal so that *new* VM provisioning is only allowed against this release (older releases like`v0.0.25` continue to run already-provisioned VMs but cannot be selected for fresh `vm create`). Both the TDX rootfs (`rootfs-prod-v0.0.27-alpha.1-tdx.iso`, sha256 `1d638b06…`) and the SEV rootfs (`rootfs-prod-v0.0.27-alpha.1-sev.iso`, sha256 `f44141c9…`) are pinned.
493
+
**Current pin (2026-05-23):**`v0.0.27`. This is the latest **non-prerelease** Production release on [GitHub](https://github.com/scrtlabs/secret-vm-build/releases/tag/v0.0.27). Both TDX and SEV verify registries publish matching `prod` entries. The SecretVM portal UI may lag (e.g. still showing`v0.0.26-beta.1`) until SCRT rolls the cluster — CI/manifests pin to v0.0.27 regardless. TDX rootfs (`rootfs-prod-v0.0.27-tdx.iso`, sha256 `1a5d9ea4…`) and SEV rootfs (`rootfs-prod-v0.0.27-sev.iso`, sha256 `17122d51…`) are pinned in `secretvm.env`.
494
494
495
495
**Upgrade procedure:** When SCRT Labs publishes a new release:
496
496
@@ -500,7 +500,7 @@ This is a **release cadence** concern, not a clock-based TTL. A version doesn't
500
500
4. Pin those SHA-256s back into `secretvm.env` and re-push.
501
501
5. Confirm the SEV registry (`scrtlabs/secretvm-verify/artifacts_registry/sev.json`) has been updated with the new release's `(vm_type, artifacts_ver)` entry — without it the SEV measurement compute will fail with `no SEV registry entry found`. SCRT Labs typically publishes the registry alongside the build artifacts; check by `curl ${SECRETVM_SEV_REGISTRY_URL} | jq '.[] | select(.artifacts_ver == "${SECRETVM_RELEASE}")'`.
502
502
503
-
**Version detection:** There is no SCRT Labs push notification or dedicated API for new releases. The GitHub Releases API (`https://api.github.com/repos/scrtlabs/secret-vm-build/releases`) can be polled, but newer versions are often marked as **pre-release** on GitHub while already required by the portal (e.g.,`v0.0.27-alpha.1` is a pre-release on GitHub but is the *only* version the portal accepts for new VMs). The `/releases/latest` endpoint only returns non-prerelease versions and should NOT be relied on. Future consideration: a scheduled GitHub Action that polls for new releases AND new SEV-registry entries, opens an issue or PR.
503
+
**Version detection:** There is no SCRT Labs push notification or dedicated API for new releases. Poll the GitHub Releases API (`https://api.github.com/repos/scrtlabs/secret-vm-build/releases`) and cross-check `secretvm-verify/artifacts_registry/{tdx.csv,sev.json}` for matching `(vm_type, artifacts_ver)` entries. Note: the portal UI can lag behind GitHub — `/releases/latest` returns the stable tag (currently`v0.0.27`) but the portal cluster may still provision older artifacts until SCRT rolls forward. Future consideration: a scheduled GitHub Action that polls for new releases AND registry entries, opens an issue or PR.
504
504
505
505
### 6.5 Auto-deploy to SecretVM test instance — DONE
506
506
@@ -827,7 +827,7 @@ See also: [`proxy-router/docs/tee-backend-verification.md`](../proxy-router/docs
827
827
| 1.13c | Variablize rootfs config — no hardcoded filenames in pipeline | S |**DONE**|`SECRETVM_ROOTFS_VARIANT` flows to cache key, download, RTMR3 compute, attestation manifest |
828
828
| 1.13d | Switch rootfs from `dev` to `prod` variant (v0.0.25) | S |**DONE**| SecretVM runs "environment prod"; `dev` rootfs produces wrong RTMR3 |
829
829
| 1.13e | Document upgrade procedure for SecretVM releases | S |**DONE**|`docs/02.3-proxy-router-tee.md` § "Upgrading SecretVM Artifacts" |
830
-
| 1.13f | Pin SecretVM release `v0.0.27-alpha.1` (TDX + SEV rootfs) | S |**DONE**|Portal-required release; this PR. SHA-256s pinned in `secretvm.env`|
830
+
| 1.13f | Pin SecretVM release `v0.0.27` (TDX + SEV rootfs) | S |**DONE**|GitHub stable prod release; SHA-256s pinned in `secretvm.env`|
831
831
| 1.13g | SEV-SNP measurement in CI: `compute-sev-measurement.py` + per-template publish | M |**DONE**| This PR; parity-tested against Go `sev_gctx.go`. See §6.3 |
832
832
| 1.13h | Bake `LOG_LEVEL_*` into `Dockerfile.tee` and surface them in `baked_env`| S |**DONE**| This PR; pipeline hard-fails if `LOG_LEVEL_APP=debug`|
833
833
| 1.14 | Full `reproduce-mr` for RTMR0-2 (Intel TDX) | M | TODO | Blocked: ACPI templates from SCRT Labs (not on critical path — RTMR0-2 verified at runtime via artifact registry lookup) |
@@ -929,7 +929,7 @@ See also: [`proxy-router/docs/tee-backend-verification.md`](../proxy-router/docs
Copy file name to clipboardExpand all lines: .ai-docs/TEE_CICD_Supply_Chain_Hardening.md
+13-13Lines changed: 13 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,10 +1,10 @@
1
1
# CI/CD Supply-Chain Hardening for Morpheus Docker Images
2
2
3
-
**Last updated:** 2026-04-29
3
+
**Last updated:** 2026-05-23
4
4
**First successful run (Phase 1a — signing):**[#22920492249](https://github.com/MorpheusAIs/Morpheus-Lumerin-Node/actions/runs/22920492249)
5
5
**First end-to-end run (Phase 1b — deploy + verify):**[#22969993910](https://github.com/MorpheusAIs/Morpheus-Lumerin-Node/actions/runs/22969993910)
6
6
7
-
> **v7.x release status (with AMD SEV-SNP).** The CI/CD hardening described here is the foundation that every downstream trust check depends on. Both **Phase 1c** (consumer-side proxy-router verification of the P-Node) and **Phase 2** (P-Node verifies its own backend LLM) have shipped on top of it — see [`TEE_Attestation_Architecture.md`](TEE_Attestation_Architecture.md) §7.4 and §7.7. **As of this release the pipeline now also publishes AMD SEV-SNP launch-digest goldens (one per portal template), pins SecretVM `v0.0.27-alpha.1` for both TDX and SEV rootfs, and bakes the proxy-router's privacy-sensitive `LOG_LEVEL_*` fields into the cosign-signed manifest's `baked_env` block.** The new SEV path mirrors the TDX one — Python compute script in CI (`compute-sev-measurement.py`) backed by the runtime Go source-of-truth (`sev_gctx.go`) with a parity test gating drift between the two.
7
+
> **v7.x release status (with AMD SEV-SNP).** The CI/CD hardening described here is the foundation that every downstream trust check depends on. Both **Phase 1c** (consumer-side proxy-router verification of the P-Node) and **Phase 2** (P-Node verifies its own backend LLM) have shipped on top of it — see [`TEE_Attestation_Architecture.md`](TEE_Attestation_Architecture.md) §7.4 and §7.7. **As of this release the pipeline now also publishes AMD SEV-SNP launch-digest goldens (one per portal template), pins SecretVM `v0.0.27` for both TDX and SEV rootfs, and bakes the proxy-router's privacy-sensitive `LOG_LEVEL_*` fields into the cosign-signed manifest's `baked_env` block.** The new SEV path mirrors the TDX one — Python compute script in CI (`compute-sev-measurement.py`) backed by the runtime Go source-of-truth (`sev_gctx.go`) with a parity test gating drift between the two.
8
8
9
9
---
10
10
@@ -137,19 +137,19 @@ This is the most important new artifact. It's a signed JSON document that record
137
137
"measurements": {
138
138
"intel_tdx": {
139
139
"rtmr3": "<96-char-hex — computed from sha256(compose) + sha256(rootfs)>",
140
-
"secretvm_release": "v0.0.27-alpha.1",
140
+
"secretvm_release": "v0.0.27",
141
141
"rootfs_variant": "rootfs-prod",
142
-
"rootfs_sha256": "<sha256 of rootfs-prod-v0.0.27-alpha.1-tdx.iso>",
142
+
"rootfs_sha256": "<sha256 of rootfs-prod-v0.0.27-tdx.iso>",
143
143
"note": "RTMR3 measures (compose + rootfs); template-independent. RTMR0-2 verified at runtime via SecretVM TDX artifact registry lookup."
144
144
},
145
145
"amd_sev_snp": {
146
146
"vcpu_type": "EPYC",
147
147
"vm_type": "prod",
148
-
"secretvm_release": "v0.0.27-alpha.1",
149
-
"rootfs_sha256": "<sha256 of rootfs-prod-v0.0.27-alpha.1-sev.iso>",
150
-
"kernel_hash": "<sha256 of bzImage-v0.0.27-alpha.1-sev>",
151
-
"initrd_hash": "<sha256 of initramfs-v0.0.27-alpha.1-sev.cpio.gz>",
152
-
"ovmf_hash": "<sha384 of ovmf-v0.0.27-alpha.1-sev.fd, registry value>",
148
+
"secretvm_release": "v0.0.27",
149
+
"rootfs_sha256": "<sha256 of rootfs-prod-v0.0.27-sev.iso>",
150
+
"kernel_hash": "<sha256 of bzImage-v0.0.27-sev>",
151
+
"initrd_hash": "<sha256 of initramfs-v0.0.27-sev.cpio.gz>",
152
+
"ovmf_hash": "<sha384 of ovmf-v0.0.27-sev.fd, registry value>",
@@ -300,10 +300,10 @@ This CI/CD hardening is the **foundation layer** for the full TEE attestation lo
300
300
| File | Change |
301
301
|---|---|
302
302
|`.github/workflows/build.yml`| Cosign signing, digest capture, SBOM, attestation manifest, RTMR3 computation, **SEV-SNP measurement compute (per template) + SEV registry fetch + baked-loglevel extraction with privacy hard-fail**, auto-deploy, and post-deploy verification. Also: GitHub Actions upgraded to Node 24-compatible versions, Go version updated to 1.25.x. |
303
-
|`.github/tee/secretvm.env`| Pins SecretVM release version (currently `v0.0.27-alpha.1`), rootfs variant (`rootfs-prod`), **TDX rootfs URL/SHA-256 AND SEV rootfs URL/SHA-256, plus the SEV artifact registry URL and registry vm_type**. All pipeline rootfs references derive from this file. |
303
+
|`.github/tee/secretvm.env`| Pins SecretVM release version (currently `v0.0.27`), rootfs variant (`rootfs-prod`), **TDX rootfs URL/SHA-256 AND SEV rootfs URL/SHA-256, plus the SEV artifact registry URL and registry vm_type**. All pipeline rootfs references derive from this file. |
304
304
|`proxy-router/scripts/compute-rtmr3.py`| Standalone RTMR3 computation script matching the `reproduce-mr` algorithm. Can be run locally for independent verification. |
305
305
|`proxy-router/scripts/compute-sev-measurement.py`|**NEW** — Standalone SEV-SNP launch-digest computation script. Byte-for-byte port of `attestation/sev_gctx.go::CalcSevMeasurement`. Computes one measurement per vCPU template (small/medium/large/2xlarge/4xlarge). Parity-tested against the runtime Go via `internal/attestation/sev_python_parity_test.go`. |
306
-
|`proxy-router/internal/attestation/sev_python_parity_test.go`|**NEW** — Hermetic regression guard: runs `compute-sev-measurement.py` as a subprocess against the v0.0.27-alpha.1 prod registry fixture and asserts byte-for-byte match against `CalcSevMeasurement` for all 5 templates. Skips gracefully when `python3` is unavailable. |
306
+
|`proxy-router/internal/attestation/sev_python_parity_test.go`|**NEW** — Hermetic regression guard: runs `compute-sev-measurement.py` as a subprocess against the v0.0.27 prod registry fixture and asserts byte-for-byte match against `CalcSevMeasurement` for all 5 templates. Skips gracefully when `python3` is unavailable. |
307
307
|`proxy-router/internal/attestation/golden.go`| Renamed JSON tag `amd_sev` → `amd_sev_snp` to align with the manifest schema; added `SEVMeasurements.PerTemplate map[string]string`, `GoldenValues.SEVPerTemplate`, and the `MatchSEVMeasurement` helper that looks up the golden by template-keyed live measurement. |
308
308
|`proxy-router/Dockerfile.tee`| Bakes immutable ENV config into the TEE image. Blockchain values are parameterized via `ARG` with mainnet defaults; overridden via `--build-arg` for testnet builds. **Logging values (`LOG_LEVEL_APP=info`, `LOG_LEVEL_TCP=warn`, `LOG_LEVEL_ETH_RPC=warn`, `LOG_LEVEL_STORAGE=warn`, plus `LOG_COLOR=false`, `LOG_JSON=true`, `LOG_IS_PROD=true`) are baked here and surfaced into the cosign-signed manifest's `baked_env` block — the privacy gate ensures the live TEE image cannot silently revert to `debug`-level app logging.**|
309
309
|`proxy-router/docker-compose.tee.yml`| Canonical compose template for TEE deployment with 5 runtime secrets. |
@@ -345,11 +345,11 @@ This CI/CD hardening is the **foundation layer** for the full TEE attestation lo
345
345
|**`GET /v1/models/attestation`**| Per-model attestation state endpoint for monitoring and forensics |**Done** — PR #699|
### Completed (this PR — SEV-SNP and v0.0.27-alpha.1 cutover)
348
+
### Completed (this PR — SEV-SNP and v0.0.27 cutover)
349
349
350
350
| Step | Description | Status |
351
351
|---|---|---|
352
-
|**SecretVM `v0.0.27-alpha.1` pin**| TDX + SEV rootfs URLs and SHA-256s pinned in `.github/tee/secretvm.env` (the portal currently only allows new VM provisioning on this release) |**Done**|
352
+
|**SecretVM `v0.0.27` pin**| TDX + SEV rootfs URLs and SHA-256s pinned in `.github/tee/secretvm.env` (GitHub latest stable Production release; portal UI may lag) |**Done**|
353
353
|**SEV rootfs download + SHA-256 verify**|`Download SecretVM rootfs (TDX + SEV)` step pulls both ISOs; both fail-closed if the pinned SHA doesn't match |**Done**|
354
354
|**SEV artifact registry fetch**|`Fetch SecretVM SEV artifact registry` step downloads `sev.json` from `scrtlabs/secretvm-verify` and records its SHA-256 in the manifest |**Done**|
0 commit comments