Skip to content

Commit a293895

Browse files
committed
auth-simple: require explicit KMS MR allowlist
1 parent b323f4f commit a293895

10 files changed

Lines changed: 548 additions & 17 deletions

File tree

docs/auth-simple-operations.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ The config is re-read on each request, so changes take effect immediately withou
3636
}
3737
```
3838

39-
> **Note:** Only `osImages` is required. Add `gatewayAppId` after deploying the Gateway. Add `apps` entries as you deploy applications.
39+
> **Note:** `osImages` is always required. For KMS authorization, you must also populate `kms.mrAggregated`; if it is left empty, auth-simple denies all KMS boots. Add `gatewayAppId` after deploying the Gateway. Add `apps` entries as you deploy applications.
4040
4141
---
4242

@@ -240,7 +240,7 @@ The `mrAggregated` is sent by the booting KMS in its auth request. To get this v
240240
KMS boot auth request: { osImageHash: '0x...', mrAggregated: '0x...', ... }
241241
```
242242

243-
2. **Initial setup**: Leave `kms.mrAggregated` empty for the first KMS (empty array allows any). After it boots, check the logs and add the value.
243+
2. **Initial setup**: capture the first KMS measurement with `Onboard.GetAttestationInfo` or from auth logs, then add it to `kms.mrAggregated` before bootstrap. An empty array now denies all KMS boots.
244244

245245
### Add to Config
246246

docs/deployment.md

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -98,13 +98,16 @@ Start in separate terminals:
9898
For production, deploy KMS and Gateway as CVMs with hardware-rooted security. Production deployments require:
9999
- KMS running in a CVM (not on the host)
100100
- Auth server for authorization (webhook mode)
101+
- KMS measurements allowlisted before bootstrap / onboarding / trusted RPCs can succeed
102+
103+
If you skip the KMS allowlist step, the VM may boot and the onboard UI may still appear, but the KMS will reject bootstrap, onboarding, or later trusted RPCs with authorization errors.
101104

102105
### Production Checklist
103106

104107
**Required:**
105108

106109
1. Set up TDX host with dstack-vmm
107-
2. Deploy KMS as CVM (with auth server)
110+
2. Deploy KMS as CVM (with auth server, capture its attestation info, and allowlist the KMS `mrAggregated` before bootstrap)
108111
3. Deploy Gateway as CVM
109112

110113
**Optional Add-ons:**
@@ -197,11 +200,16 @@ Create `auth-config.json` for initial KMS deployment:
197200
```json
198201
{
199202
"osImages": ["0x<os-image-hash>"],
200-
"kms": { "allowAnyDevice": true },
203+
"kms": {
204+
"mrAggregated": ["0x<kms-mr-aggregated>"],
205+
"allowAnyDevice": true
206+
},
201207
"apps": {}
202208
}
203209
```
204210

211+
> **Important:** `auth-simple` now treats an empty `kms.mrAggregated` allowlist as deny-all for KMS. Capture the current KMS measurement with `Onboard.GetAttestationInfo` and add it before bootstrap.
212+
205213
Run auth-simple:
206214

207215
```bash
@@ -459,7 +467,7 @@ Additional KMS instances can onboard from an existing KMS to share the same root
459467
```toml
460468
[core.onboard]
461469
enabled = true
462-
auto_bootstrap_domain = "" # Empty = onboard mode # Require TDX attestation
470+
auto_bootstrap_domain = "" # Empty = onboard mode
463471
address = "0.0.0.0"
464472
port = 9203 # HTTP port for onboard UI
465473
```
@@ -479,7 +487,12 @@ curl http://<new-kms>:9203/finish
479487
# Restart KMS - it will now serve as a full KMS with shared keys
480488
```
481489

482-
> **Note:** KMS onboarding requires attested KMS instances. Add the KMS `mrAggregated` hash to your auth server's `kms.mrAggregated` whitelist.
490+
> **Note:** KMS onboarding requires attested KMS instances, and both sides must already be authorized. Add the relevant KMS `mrAggregated` hashes to your auth backend first:
491+
>
492+
> - the destination KMS must allow the source KMS
493+
> - the source KMS must allow the destination KMS
494+
>
495+
> If you skip this, `Onboard.Onboard` or later trusted RPCs will fail with KMS authorization errors.
483496
484497
---
485498

docs/tutorials/kms-cvm-deployment.md

Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -313,20 +313,49 @@ Onboarding
313313
```
314314

315315
> **Important:** KMS is now in onboard mode — a plain HTTP server waiting for bootstrap. It will **not** serve TLS or respond to `KMS.GetMeta` until you complete the next step.
316+
>
317+
> **Critical prerequisite:** before bootstrap can succeed, the KMS must already be authorized by your auth backend.
318+
>
319+
> - For `auth-simple`, add the KMS `mrAggregated` to `kms.mrAggregated`
320+
> - For `auth-eth`, add the KMS `mrAggregated` on-chain with `addKmsAggregatedMr(...)`
321+
>
322+
> You can fetch the value before bootstrap with:
323+
>
324+
> ```bash
325+
> curl -s -X POST \
326+
> -H "Content-Type: application/json" \
327+
> -d '{}' \
328+
> "http://localhost:9100/prpc/Onboard.GetAttestationInfo?json" | jq .
329+
> ```
330+
>
331+
> If you skip this step, `Onboard.Bootstrap` will fail with a KMS authorization error and the KMS will not enter normal service.
332+
>
333+
> **Pre-bootstrap checklist:**
334+
>
335+
> 1. `Onboard.GetAttestationInfo` returns the current KMS measurement
336+
> 2. that `mrAggregated` has been allowlisted in your auth backend
337+
> 3. the auth backend is reachable from the KMS CVM
338+
> 4. you are still calling the onboard HTTP endpoint, not the post-bootstrap TLS endpoint
316339
317340
### Step 6: Bootstrap KMS
318341
319342
With KMS in onboard mode, trigger key generation by calling the Bootstrap RPC endpoint. This generates root keys, a TDX attestation quote, and writes `bootstrap-info.json`:
320343
321344
```bash
345+
# Inspect the KMS measurement before bootstrap
346+
curl -s -X POST \
347+
-H "Content-Type: application/json" \
348+
-d '{}' \
349+
"http://localhost:9100/prpc/Onboard.GetAttestationInfo?json" | jq .
350+
322351
# Replace kms.yourdomain.com with your actual KMS domain
323352
curl -s -X POST \
324353
-H "Content-Type: application/json" \
325354
-d '{"domain":"kms.yourdomain.com"}' \
326355
"http://localhost:9100/prpc/Onboard.Bootstrap?json" | tee ~/kms-deploy/bootstrap-info.json | jq .
327356
```
328357
329-
> **Note:** This uses plain `http://` — KMS is still in onboard mode (no TLS yet). The `tee` command saves the response to `bootstrap-info.json` while also displaying it. You'll need this file later to register KMS on-chain.
358+
> **Note:** This uses plain `http://` — KMS is still in onboard mode (no TLS yet). The `tee` command saves the response to `bootstrap-info.json` while also displaying it. You'll need this file later to register KMS on-chain. If this call fails with a KMS authorization error, allowlist the `mrAggregated` value first and retry.
330359
331360
Expected response:
332361

kms/auth-simple/README.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20,12 +20,13 @@ bun install
2020

2121
Create `auth-config.json` (see `auth-config.example.json`).
2222

23-
For initial KMS deployment, you only need the OS image hash:
23+
For KMS deployment, you must allowlist both the OS image hash and the KMS `mrAggregated` value:
2424

2525
```json
2626
{
2727
"osImages": ["0x0b327bcd642788b0517de3ff46d31ebd3847b6c64ea40bacde268bb9f1c8ec83"],
2828
"kms": {
29+
"mrAggregated": ["0x<kms-mr-aggregated>"],
2930
"allowAnyDevice": true
3031
},
3132
"apps": {}
@@ -39,7 +40,7 @@ Add more fields as you deploy Gateway and apps:
3940
"osImages": ["0x..."],
4041
"gatewayAppId": "0x...",
4142
"kms": {
42-
"mrAggregated": [],
43+
"mrAggregated": ["0x..."],
4344
"devices": [],
4445
"allowAnyDevice": true
4546
},
@@ -59,7 +60,7 @@ Add more fields as you deploy Gateway and apps:
5960
|-------|----------|-------------|
6061
| `osImages` | Yes | Allowed OS image hashes (from `digest.txt`) |
6162
| `gatewayAppId` | No | Gateway app ID (add after Gateway deployment) |
62-
| `kms.mrAggregated` | No | Allowed KMS aggregated MR values |
63+
| `kms.mrAggregated` | Yes for KMS authorization | Allowed KMS aggregated MR values. An empty array denies all KMS boots. |
6364
| `kms.devices` | No | Allowed KMS device IDs |
6465
| `kms.allowAnyDevice` | No | If true, skip device ID check for KMS |
6566
| `apps.<appId>.composeHashes` | No | Allowed compose hashes for this app |
@@ -160,7 +161,7 @@ KMS boot authorization.
160161

161162
1. `tcbStatus` must be "UpToDate"
162163
2. `osImageHash` must be in `osImages` array
163-
3. `mrAggregated` must be in `kms.mrAggregated` (if non-empty)
164+
3. `mrAggregated` must be in `kms.mrAggregated`
164165
4. `deviceId` must be in `kms.devices` (unless `allowAnyDevice` is true)
165166

166167
### App Boot Validation

kms/auth-simple/auth-config.example.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,9 @@
33
"0x0b327bcd642788b0517de3ff46d31ebd3847b6c64ea40bacde268bb9f1c8ec83"
44
],
55
"kms": {
6-
"mrAggregated": [],
6+
"mrAggregated": [
7+
"0x<kms-mr-aggregated>"
8+
],
79
"devices": [],
810
"allowAnyDevice": true
911
},

kms/auth-simple/bun.lock

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

kms/auth-simple/index.test.ts

Lines changed: 29 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,10 @@ describe('auth-simple', () => {
7575
writeTestConfig({
7676
gatewayAppId: '0xgateway',
7777
osImages: ['0x1fbb0cf9cc6cfbf23d6b779776fabad2c5403d643badb9e5e238615e4960a78a'],
78-
kms: { allowAnyDevice: true }
78+
kms: {
79+
mrAggregated: ['0xabc123'],
80+
allowAnyDevice: true
81+
}
7982
});
8083

8184
const res = await app.fetch(new Request('http://localhost/bootAuth/kms', {
@@ -93,7 +96,10 @@ describe('auth-simple', () => {
9396
writeTestConfig({
9497
gatewayAppId: '0xgateway',
9598
osImages: ['0xdifferentimage'],
96-
kms: { allowAnyDevice: true }
99+
kms: {
100+
mrAggregated: ['0xabc123'],
101+
allowAnyDevice: true
102+
}
97103
});
98104

99105
const res = await app.fetch(new Request('http://localhost/bootAuth/kms', {
@@ -128,6 +134,27 @@ describe('auth-simple', () => {
128134
expect(json.reason).toContain('MR');
129135
});
130136

137+
it('rejects KMS boot when the allowlist is empty', async () => {
138+
writeTestConfig({
139+
gatewayAppId: '0xgateway',
140+
osImages: ['0x1fbb0cf9cc6cfbf23d6b779776fabad2c5403d643badb9e5e238615e4960a78a'],
141+
kms: {
142+
mrAggregated: [],
143+
allowAnyDevice: true
144+
}
145+
});
146+
147+
const res = await app.fetch(new Request('http://localhost/bootAuth/kms', {
148+
method: 'POST',
149+
headers: { 'Content-Type': 'application/json' },
150+
body: JSON.stringify(baseBootInfo)
151+
}));
152+
const json = await res.json();
153+
154+
expect(json.isAllowed).toBe(false);
155+
expect(json.reason).toContain('MR');
156+
});
157+
131158
it('allows KMS boot with allowAnyDevice', async () => {
132159
writeTestConfig({
133160
gatewayAppId: '0xgateway',

kms/auth-simple/index.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,7 @@ class ConfigBackend {
122122

123123
// check aggregated MR
124124
const allowedMrs = config.kms.mrAggregated.map(normalizeHex);
125-
if (allowedMrs.length > 0 && !allowedMrs.includes(mrAggregated)) {
125+
if (!allowedMrs.includes(mrAggregated)) {
126126
return {
127127
isAllowed: false,
128128
reason: 'aggregated MR not allowed',

0 commit comments

Comments
 (0)