Skip to content

Support non-CC runtimes for development#2337

Closed
msanft wants to merge 13 commits intomainfrom
msanft/dev-runtime
Closed

Support non-CC runtimes for development#2337
msanft wants to merge 13 commits intomainfrom
msanft/dev-runtime

Conversation

@msanft
Copy link
Copy Markdown
Member

@msanft msanft commented Apr 21, 2026 •
edited
Loading

This adds support for insecure non-CC runtimes. These can be used both for testing in PoC-like settings where Contrast needs to be integrated into a given platform where CC hardware is not yet available, but also eases benchmarking of CC against non-CC performance for Contrast consumers.

This PR only adds the technical implementation for the insecure runtimes, but doesn't yet expose it to the user via release artifacts or documentation.

Despite the PR size, I opted to combine the changes into a single PR with reviewable commits, since -imo- having the full test coverage of all the code makes it easier to reason about the treewide changes. Let me know if you want anything adjusted.

Closes CON-2, CON-3, CON-4

@msanft msanft requested review from burgerdev and charludo April 21, 2026 09:54
@msanft msanft added the no changelog PRs not listed in the release notes label Apr 21, 2026
@msanft msanft force-pushed the msanft/dev-runtime branch from d4587b9 to e026aae Compare April 21, 2026 10:00
@msanft
Copy link
Copy Markdown
Member Author

msanft commented Apr 21, 2026 •
edited
Loading

Insecure E2E Test

@msanft msanft force-pushed the msanft/dev-runtime branch 3 times, most recently from 7f4899c to 4249f55 Compare April 21, 2026 10:15
sespiros
sespiros reviewed Apr 21, 2026
View reviewed changes
Comment thread cli/verifier/image_ref_valid.go Outdated
@msanft msanft requested a review from sespiros April 21, 2026 15:56
@msanft msanft force-pushed the msanft/dev-runtime branch 4 times, most recently from 83b5110 to 66bf651 Compare April 22, 2026 07:39
@msanft
Copy link
Copy Markdown
Member Author

msanft commented Apr 23, 2026 •
edited
Loading

TODO: Handling of insecure runtimes in the SDK
For Privatemode, this is important as all the attestation parts needed for a CC/Non-CC benchmark are based on the SDK.

EDIT: Changed this to match the CLI's opt-in behavior.

@msanft msanft force-pushed the msanft/dev-runtime branch from a475695 to d289cdd Compare April 24, 2026 08:11
msanft added 13 commits April 24, 2026 14:47
@msanft
platforms: add insecure platforms
16f00be 
This adds definitions for the insecure platform variants.
@msanft
nix: add reference values for insecure platforms
b8be345 
This adds stub reference values for the insecure development platforms
to the JSON file with all the reference values.
@msanft
cli: allow generation for insecure platforms
b065c10 
This adds support for insecure platforms to `contrast generate`.
To enable generation for insecure platforms, both the `--INSECURE` flag
and the `CONTRAST_ALLOW_INSECURE_RUNTIMES` environment variable need to
be set. The `contrast-cc-*` and `contrast-insecure-*` runtimes can be
used interchangeably. The bare `contrast-cc` and `contrast-insecure` only
resolve if `--reference-values` is set to a CC/insecure platform,
respectively.
@msanft
justfile: enable development for insecure runtimes
2b35ed7 
This adjusts the justfile to include insecure runtimes. Additionally, it
makes the `just runtime` target *overwrite* the runtimes instead of
appending them. The latter caused issues when the file already had
content from a previous run, and I don't see any use for that behaviour.
@msanft
nodeinstaller: support insecure platforms
a64e748 
This adds the node deployment step for the insecure platforms. It uses
the same Kata configuration as the CC variants, but strips the CC-
enabling values from there.
@msanft
initdata-processor: skip validation on insecure platforms
2b4556f
@msanft
attestation: add insecure issuer / validator
e08d319 
This adds an insecure attestation issuer / validator implementation.
This implementation just forwards the hostdata-provided initdata digest
to the validator. The scaffolding necessary for this is optionally
making the initdata-processor long-running to expose the initdata
digest via an HTTP server, as the configfs-backed hostdata channel is
not available on insecure platforms.
@msanft
coordinator: make insecure platforms opt-in at generation time
5fa112b 
This requires the coordinator to be initially configured to allow
insecure runtimes by requiring the `CONTRAST_ALLOW_INSECURE` environment
variable to be set within the coordinator. This prevents deploying an
insecure manifest to a previously-secure runtime, which is a use-case
we don't need to support.
@msanft
cli: require opt-in for verifying insecure deployments
452f9ac 
This adds the same `--INSECURE` and `CONTRAST_ALLOW_INSECURE_RUNTIMES`
toggles to `contrast verify` that are already present in
`contrast generate`.
@msanft
e2e: add insecure deployment test
d99b93a 
This adds a basic E2E test for the insecure runtimes.
@msanft
treewide: apply deMorgan rule
36e9a9b
@msanft
kuberesource: increase overhead for insecure platforms
3f16d83 
With insecure GPU VMs, we ran into OOM kills on larger VM sizes.
The theory is that this is caused by IOMMU page tables which are only
allocated when VFIO passthrough is used. In the CC case, these page
tables get allocated under the TDX modules' memory. For the non-CC
case, they are part of the QEMU process, and thus contribute to the
memory usage within the Kubelet's cgroup. To counter this, we add a
little bit of extra overhead for insecure GPU VMs.
@msanft
sdk: require opt-in for insecure deployments.
4b54b0e 
For verifying an insecure deployment, an SDK user must now instantiate
the client `.WithInsecure`, aligning it to the CLI behavior.
@msanft msanft force-pushed the msanft/dev-runtime branch from d289cdd to 4b54b0e Compare April 24, 2026 12:47
burgerdev
burgerdev requested changes Apr 27, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@burgerdev burgerdev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @msanft,

Thanks for the PR. I think this could be split up naturally into at least the following intermediate features which don't need to be submitted all at once:

  • prefactoring for the contrast-cc runtime class name matching.
  • Insecure issuer and verifier, not wired into anything but tested against each other.
  • initdata processor
  • runtime and nodeinstaller changes
  • support for insecure manifests at the Coordinator
  • support for insecure manifests at the CLI
  • e2e test with insecure
  • support for insecure manifests in SDK

That should make review (and iteration) much easier - thanks!

@msanft
Copy link
Copy Markdown
Member Author

msanft commented Apr 27, 2026 •
edited
Loading

That should make [...] iteration much easier - thanks!

@burgerdev: We'll have to find a good working mode for this. For me, as the PR author, rebasing a stack of N PRs is very tedious. That was my rationale for bundling these changes into a single PR and relying on reviewable commits instead. I'll try to split up into multiple PRs as you suggested.

This was referenced Apr 27, 2026
Merged
Merged
Open
Merged
Open
Open
Open
Open
@msanft
Copy link
Copy Markdown
Member Author

msanft commented Apr 27, 2026 •
edited
Loading

I split this with the help of AI:

--- START AGENT OUTPUT

  1. [1/8] prefactor: centralize Contrast runtime class matching #2352 prefactor: centralize Contrast runtime class matching
  2. [2/8] attestation: add insecure issuer and validator #2353 attestation: add insecure issuer and validator
  3. [4/8] initdata-processor: support insecure platforms #2354 initdata-processor: support insecure platforms
  4. [4/8] runtime/nodeinstaller: add insecure platform variants #2355 runtime/nodeinstaller: add insecure platform variants
  5. [5/8] coordinator: support insecure manifests behind opt-in #2356 coordinator: support insecure manifests behind opt-in
  6. [6/8] cli: support insecure manifests behind opt-in #2357 cli: support insecure manifests behind opt-in
  7. [7/8] e2e: add insecure deployment test #2358 e2e: add insecure deployment test
  8. [8/8] sdk: support insecure manifests behind opt-in #2359 sdk: support insecure manifests behind opt-in

Each PR is based on the previous one.

--- END AGENT OUTPUT

@msanft msanft closed this Apr 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@burgerdev burgerdev burgerdev requested changes

@charludo charludo Awaiting requested review from charludo

@sespiros sespiros Awaiting requested review from sespiros

Assignees

No one assigned

Labels

no changelog PRs not listed in the release notes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@msanft @sespiros @burgerdev