|
| 1 | +# GPU Attestation Collateral Proxy |
| 2 | + |
| 3 | +Local GPU attestation (`nvattest --verifier local`) makes live calls to two |
| 4 | +NVIDIA services at every CVM boot: the OCSP responder |
| 5 | +(`ocsp.ndis.nvidia.com`, one request per certificate in three chains) and the |
| 6 | +RIM service (`rim.attestation.nvidia.com`, driver + VBIOS RIM documents). |
| 7 | +Because the GPU attestation gate fails closed, anything that breaks those |
| 8 | +calls — an egress-restricted or air-gapped network, an upstream outage, or a |
| 9 | +middlebox — keeps every GPU CVM from booting. |
| 10 | + |
| 11 | +`gpu-attest-proxy` is a small host-side (or site-level) caching proxy that |
| 12 | +removes the guest's direct dependency on those services, in the same spirit |
| 13 | +as Intel's PCCS for TDX collateral: |
| 14 | + |
| 15 | +- **OCSP** requests are relayed byte-for-byte; successful responses are |
| 16 | + cached keyed by CertID until their `nextUpdate`, so repeat boots and |
| 17 | + fleets of CVMs sharing GPUs stop hitting the responder. |
| 18 | +- **RIM** documents are cached by RIM id and re-fetched on a TTL; when the |
| 19 | + upstream is unreachable, a stale document keeps being served within a |
| 20 | + bounded window. |
| 21 | +- A background refresher renews entries before they expire, so a warm cache |
| 22 | + rides through upstream outages with close to a full validity window. |
| 23 | + |
| 24 | +Everything the proxy serves is signed collateral that the guest verifies |
| 25 | +itself (OCSP response signatures and validity windows via |
| 26 | +`OCSP_basic_verify`, RIM signatures and certificate chains by the attestation |
| 27 | +SDK). The proxy needs no secrets and cannot forge responses; its worst |
| 28 | +misbehavior is withholding service, which fails closed — the same outcome as |
| 29 | +blocking NVIDIA's endpoints directly. |
| 30 | + |
| 31 | +## Why the OCSP nonce check is relaxed |
| 32 | + |
| 33 | +The attestation SDK adds a random nonce to every OCSP request and its |
| 34 | +built-in appraisal policy requires the response to echo it |
| 35 | +(`x-nvidia-cert-ocsp-nonce-matches`). A response cached for another request |
| 36 | +can never echo this request's nonce, so caching is only possible if the |
| 37 | +guest relaxes that single check. |
| 38 | + |
| 39 | +When `gpu_attest_proxy_url` is set, `dstack-util` runs nvattest with |
| 40 | +`--relying-party-policy` pointing at NVIDIA's |
| 41 | +`allow_trust_outpost_ocsp.rego` (packaged in the image at |
| 42 | +`/usr/share/nvattest/policies/`), which keeps every built-in check except |
| 43 | +the nonce match. Freshness is then bounded by the response validity window |
| 44 | +instead of the nonce: a revoked certificate may keep passing until the |
| 45 | +cached response's `nextUpdate`. This is the same trade-off the SDK's own |
| 46 | +in-process OCSP cache makes. Guests without the proxy configured keep the |
| 47 | +default policy and the nonce check. |
| 48 | + |
| 49 | +## Setup |
| 50 | + |
| 51 | +Run the proxy somewhere every CVM can reach (typically on each host, next to |
| 52 | +dstack-vmm): |
| 53 | + |
| 54 | +```console |
| 55 | +$ gpu-attest-proxy -c /etc/gpu-attest-proxy/gpu-attest-proxy.toml |
| 56 | +``` |
| 57 | + |
| 58 | +See `dstack/gpu-attest-proxy/gpu-attest-proxy.toml` for the default |
| 59 | +configuration (upstreams, cache directory, TTLs). The cache persists as one |
| 60 | +JSON file per entry under `cache_dir`, so restarts keep it warm. |
| 61 | + |
| 62 | +Point guests at it through `vmm.toml`: |
| 63 | + |
| 64 | +```toml |
| 65 | +[cvm] |
| 66 | +gpu_attest_proxy_url = "http://<host-bridge-ip>:8090" |
| 67 | +``` |
| 68 | + |
| 69 | +The VMM passes the URL to the guest via sys-config. At boot, `dstack-util` |
| 70 | +derives the SDK endpoints from it: |
| 71 | + |
| 72 | +- `--ocsp-url {base}/ocsp` |
| 73 | +- `--rim-url {base}` (the SDK appends `/v1/rim/{rim_id}` itself) |
| 74 | +- `--relying-party-policy /usr/share/nvattest/policies/allow_trust_outpost_ocsp.rego` |
| 75 | + |
| 76 | +When the value is empty (default), guests talk to NVIDIA directly and the |
| 77 | +built-in appraisal policy applies unchanged. |
| 78 | + |
| 79 | +## Operational notes |
| 80 | + |
| 81 | +- `GET /health` and `GET /info` expose liveness and cache statistics. |
| 82 | +- A cold cache cannot help: the first boot after an outage begins still |
| 83 | + needs a reachable upstream. Keep the proxy running so entries stay warm; |
| 84 | + `refresh_margin` controls how far ahead of expiry entries are renewed. |
| 85 | +- The guest-side clock check is unchanged: OCSP validity windows are |
| 86 | + verified by the guest, so an expired cached response never passes. |
| 87 | +- Revocation visibility degrades from "immediate" (nonce) to the OCSP |
| 88 | + response validity window (`nextUpdate`). Measure the actual window for |
| 89 | + your certificate chains — it determines both how long an outage the cache |
| 90 | + can absorb and how long a revocation can go unnoticed. |
0 commit comments