-
Notifications
You must be signed in to change notification settings - Fork 157
feat: add OCI 1.1 Referrers API support with configurable distribution #1691
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
anithapriyanatarajan
wants to merge
1
commit into
tektoncd:main
Choose a base branch
from
anithapriyanatarajan:oci-referrer-api-enablement
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
236 changes: 236 additions & 0 deletions
236
docs/oci-artifact-distribution-format-referrers-schema.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,236 @@ | ||
| <!-- | ||
| --- | ||
| linkTitle: "OCI Artifact Distribution (Referrers)" | ||
| weight: 35 | ||
| --- | ||
| --> | ||
|
|
||
| # OCI Artifact Distribution: the Referrers Schema | ||
|
|
||
| When Chains is configured with `oci` as the artifact storage backend, it signs the resulting image and uploads the signature (`.sig`) and SLSA Provenance (`.att`) for the TaskRun or PipelineRun to the configured OCI repository. As OCI registries adopt the [OCI Distribution Spec v1.1.1](https://specs.opencontainers.org/distribution-spec/?v=v1.1.1), [PR#1691](https://github.com/tektoncd/chains/pull/1691) introduced support for uploading these blobs in the Referrers API format, when the registry supports it. This page explains the additional configuration required to adopt the new approach. | ||
|
|
||
| If you don't change anything, Chains uses the older tag-based layout and just works. Proceed reading further, if you want to use the OCI 1.1 Referrers API instead. | ||
|
|
||
| ## The problem with the old layout | ||
|
|
||
| cosign, which Chains uses under the hood, has traditionally stored a signature | ||
| or attestation by pushing an extra tag next to the image. For an image at digest | ||
| `sha256:abc...`, it creates tags like `sha256-abc....sig` and `sha256-abc....att`. | ||
|
|
||
| This works on every registry, but it has drawbacks: | ||
|
|
||
| - Tags are meant to name top-level artifacts you look up and pull. Reusing them | ||
| for supporting metadata (signatures, attestations) puts that metadata in the | ||
| same namespace as the real artifacts, where it is easy to confuse the two. | ||
| - The `.att` tag points to a single manifest that holds *all* of an image's | ||
| attestations, and that manifest has no stable digest. When another process | ||
| adds an attestation, they are folded back into the same manifest and its | ||
| digest changes, so there is no stable, individually addressable reference to a | ||
| single attestation. | ||
| - No OCI standard describes this layout, so every tool has to special-case it. | ||
|
|
||
| ## What the Referrers schema is | ||
|
|
||
| The [OCI 1.1 distribution spec](https://github.com/opencontainers/distribution-spec/blob/v1.1.0/spec.md#listing-referrers) | ||
| added a standard way to attach one artifact to another. Instead of inventing a | ||
| tag, you push the signature or attestation as its own manifest that records the | ||
| image it belongs to in a `subject` field. The registry can then answer a simple | ||
| question: "what artifacts refer to this image?" | ||
|
|
||
| This is the **Referrers schema**. The payoff is no extra tags, a clean registry, | ||
| and a standard that registries and policy tools already understand. | ||
|
|
||
| ## How cosign enables it | ||
|
|
||
| cosign and the `go-containerregistry` library it uses both understand the | ||
| Referrers schema, and Chains drives them directly through that library rather | ||
| than shelling out to the cosign CLI. When Chains writes in referrers mode, the | ||
| library prefers the registry's native Referrers API if it is available. If the | ||
| registry does not expose it, the library falls back to the spec's **referrers | ||
| tag schema**: it maintains a single `sha256-<digest>` index tag that lists the | ||
| referrers. | ||
|
|
||
| Either way it is still "referrers mode" — the stored content is the same, no | ||
| `.sig` or `.att` tags are created, and `cosign verify` and `oras discover` both | ||
| work. This fallback is automatic and needs no configuration, so it also covers | ||
| registries that don't yet have native support. | ||
|
|
||
| ## How Chains enables it | ||
|
|
||
| Chains exposes this through a single config flag in the `chains-config` | ||
| ConfigMap: | ||
|
|
||
| ```yaml | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: chains-config | ||
| namespace: tekton-chains | ||
| data: | ||
| storage.oci.distribution-method: "referrers-api" | ||
| ``` | ||
|
|
||
| | Value | What Chains does | | ||
| |---|---| | ||
| | `legacy` (default) | Old tag-based layout (`.sig` / `.att` tags), DSSE-encoded. Works everywhere. | | ||
| | `referrers-api` | OCI 1.1 Referrers schema. No extra tags, with automatic fallback on registries that lack native support. | | ||
|
|
||
| That's the only knob. You pick *where* artifacts go, and Chains picks the right | ||
| encoding to match (explained next). | ||
|
|
||
| > [!NOTE] | ||
| > This flag only takes effect when the OCI storage backend is in use — that is, | ||
| > when `artifacts.oci.storage`, `artifacts.taskrun.storage`, or | ||
| > `artifacts.pipelinerun.storage` includes `oci`. If you store signatures and | ||
| > attestations somewhere else (docstore, mongo, Grafeas or other options supported in chains), `storage.oci.distribution-method` has no effect. | ||
|
|
||
| > [!TIP] | ||
| > Chains vendors the Sigstore libraries it needs, so referrers mode works out of | ||
| > the box — there is nothing extra to install. You only need a separate `cosign` | ||
| > or `oras` CLI if you want to verify or inspect the stored artifacts yourself, | ||
| > as shown below. | ||
|
|
||
| ## Why the encoding is tied to the layout | ||
|
|
||
| The two choices line up with cosign's own defaults as they have evolved: | ||
|
|
||
| - **legacy** matches cosign's original default — tag-based storage with the | ||
| attestation written as a **DSSE** envelope. This is what existing tooling has | ||
| always verified. | ||
| - **referrers-api** writes the attestation over the OCI 1.1 Referrers API as a | ||
| **Sigstore protobuf bundle**, the format newer cosign versions write and | ||
| verify. | ||
|
|
||
| Chains deliberately ties the encoding to the layout instead of exposing it as a | ||
| separate switch. The reason is compatibility: cosign's verification for | ||
| referrer-stored attestations expects the protobuf bundle, so a | ||
| referrers-plus-DSSE attestation cannot be reliably verified with | ||
| `cosign verify-attestation`. Pairing referrers with protobuf keeps Chains in | ||
| step with cosign and avoids a matrix of combinations that no tool can verify. | ||
|
|
||
| Image **signatures** are handled differently from attestations. A cosign image | ||
| signature is a plain signature over the payload, not a DSSE envelope, so in | ||
| referrers mode Chains writes it using cosign's native signature manifest — the | ||
| exact shape `cosign verify` looks for — rather than a protobuf bundle. Signature | ||
| verification therefore works with no extra flags. | ||
|
|
||
| ## Verifying | ||
|
|
||
| Verification is the same in both modes — point cosign at your key: | ||
|
|
||
| ```shell | ||
| # Verify a signature | ||
| cosign verify \ | ||
| --key k8s://tekton-chains/signing-secrets \ | ||
| <image>@sha256:<digest> | ||
|
|
||
| # Verify an attestation | ||
| cosign verify-attestation \ | ||
| --key k8s://tekton-chains/signing-secrets \ | ||
| --type slsaprovenance \ | ||
| <image>@sha256:<digest> | ||
| ``` | ||
|
|
||
| > [!IMPORTANT] | ||
| > Starting with **cosign v2.0**, and continuing through the v3.x series, | ||
| > `cosign verify` and `cosign verify-attestation` check for transparency-log | ||
| > (Rekor) inclusion **by default**. Whether that check can pass depends on | ||
| > Chains' transparency setting: | ||
| > | ||
| > - **Transparency disabled** (`transparency.enabled: "false"`): Chains does not | ||
| > record signatures in a transparency log, so there are no Rekor entries and | ||
| > the default cosign check fails with an error such as: | ||
| > | ||
| > ```text | ||
| > Error: no matching signatures: ... not enough verified log entries from | ||
| > transparency log: 0 < 1 | ||
| > ``` | ||
| > | ||
| > This is not a signature problem. Add `--insecure-ignore-tlog=true` to the | ||
| > commands above to verify against the public key alone. | ||
| > | ||
| > - **Transparency enabled** (`transparency.enabled: "true"`): signatures are | ||
| > recorded in Rekor, so the default tlog check passes and no extra flag is | ||
| > needed. | ||
|
|
||
| To see what was stored, use [`oras`](https://oras.land/): | ||
|
|
||
| ```shell | ||
| oras discover <image>@sha256:<digest> | ||
| ``` | ||
|
|
||
| In `referrers-api` mode you will see two referrers. The signature manifest does not | ||
| have an `artifactType` set (cosign uses the layer `mediaType` instead), so `oras` | ||
| displays it as `<unknown>`. The attestation manifest carries | ||
| `application/vnd.dev.sigstore.bundle.v0.3+json` as its `artifactType`, which `oras` | ||
| displays directly: | ||
|
|
||
| ```text | ||
| <image>@sha256:<digest> | ||
| ├── <unknown> ← signature | ||
| │ └── sha256:<sig-manifest-digest> | ||
| └── application/vnd.dev.sigstore.bundle.v0.3+json ← attestation | ||
| └── sha256:<att-manifest-digest> | ||
| ``` | ||
|
|
||
| In `legacy` mode, `oras discover` returns no referrers because the signature and | ||
| attestation are stored as ordinary tags (`.sig` / `.att`) rather than referrer | ||
| manifests. | ||
|
|
||
| ## Things to keep in mind in referrers mode | ||
|
|
||
| These are interoperability notes, not bugs in Chains. | ||
|
|
||
| 1. **`storage.oci.repository` is ignored.** This setting normally redirects where | ||
| OCI signatures and attestations are stored, letting you keep them in a | ||
| different repository from the image. A referrer, by contrast, must live in the | ||
| same repository as the image it points at, because the referrer manifest | ||
| references its subject by digest within that repository. In referrers mode | ||
| Chains logs a warning and stores the referrer next to the image. The override | ||
| still works in `legacy` mode. | ||
|
|
||
| 2. **Older cosign discovery paths may not surface the attestation.** Chains | ||
| stores attestations as a protobuf bundle, which is the default for current | ||
| cosign versions. Older cosign releases that default to the tag-based layout | ||
| discover attestations by a different type and may not list it. The attestation | ||
| is still present — `oras discover` shows it and policy engines can consume it — | ||
| and `cosign verify` of the signature is unaffected. | ||
|
|
||
| 3. **Some registries accept a write but don't return it on read.** If a registry | ||
| reports success but you can't read the referrer back, it isn't fully OCI 1.1 | ||
| compliant. Switch that registry to `legacy`. | ||
|
|
||
| 4. **`oras discover` shows the signature as `<unknown>`.** cosign does not set | ||
| `artifactType` on signature manifests — it identifies signatures by the layer | ||
| `mediaType` (`application/vnd.dev.cosign.simplesigning.v1+json`) instead. As a | ||
| result, `oras` cannot identify the artifact type and falls back to `<unknown>`. | ||
| This is expected and consistent across all registries. The attestation always | ||
| shows its `artifactType` (`application/vnd.dev.sigstore.bundle.v0.3+json`) | ||
| because that is set on the manifest itself. Verification with `cosign verify` | ||
| and `cosign verify-attestation` is unaffected. | ||
|
|
||
| 5. **Concurrent writes can race on the tag-schema fallback.** On registries | ||
| without a native Referrers API, the index tag is updated with a | ||
| read-append-write cycle, so simultaneous writes to the same image can drop an | ||
| entry. Registries with the native Referrers API are not affected. This does | ||
| not apply to `legacy` mode. | ||
|
|
||
| ## Registry compatibility | ||
|
|
||
| cosign works with a wide range of registries, including Amazon ECR, Azure | ||
| Container Registry, Docker Hub, GitHub Container Registry, GitLab Container | ||
| Registry, Google Artifact Registry, Harbor, JFrog Artifactory, and Quay. See the | ||
| [cosign registry support page](https://docs.sigstore.dev/cosign/system_config/registry_support/) | ||
| for the current list. | ||
|
|
||
| Both `legacy` and `referrers-api` work against any OCI-compliant registry. | ||
| In `referrers-api` mode, registries with a native Referrers API use it directly; | ||
| the rest fall back automatically to the referrers tag schema, as described above. | ||
| You do not need to know a registry's level of OCI 1.1 support in advance. | ||
|
|
||
| ## See also | ||
|
|
||
| - [Chains configuration reference](config.md) — all `storage.oci.*` keys. | ||
| - [Signing](signing.md) — how signing keys and secrets are configured. | ||
| - [cosign registry support](https://docs.sigstore.dev/cosign/system_config/registry_support/) | ||
| - [OCI distribution spec — Listing Referrers](https://github.com/opencontainers/distribution-spec/blob/v1.1.0/spec.md#listing-referrers) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suspect that users will care more about the format whether that is legacy or sigstore bundle formats. It is just that sigstore bundles also happen to require the referrer's api. I don't know how much reframing of the document that would require.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
referrers-apiis specific to the OCI storage backend. All other backends (GCS, Firestore, Grafeas, docdb, Archivista, Tekton annotations) continue to use DSSE as-is. This PR does not change them.As long as cosign
verify-attestationand cosignverify-blob-attestationcontinue to support DSSE-encoded attestations, the current split is clean. OCI users can opt into the new bundle format via referrers-api, while everything else stays on DSSE. I've raised a clarification with the cosign/sigstore community on this point: https://sigstore.slack.com/archives/C01PZKDL4DP/p1782872895452699.However I would like to get views from @tektoncd/chains-collaborators on a broader question: should Chains eventually move all storage backends to the Sigstore protobuf bundle format (which mandates protobuf serialization), or is the current per-backend split the right long-term architecture?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@arewm - just to reconfirm, from a user's perspective, choosing
referrers-apidefaults to sigstore bundle. There is no referrer api support DSSE format. So with that clarification, I hope this PR is complete. Rest of the Backends, I prefer to handle in separate PR. Do you have any further concerns with this? Thank you.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I agree with that statement but if we expect that users' primary concern for supportability is the format (i.e. bundle vs DSSE) then we should provide that as the toggle. Many registries support the referrer's api v1.1. Again, this is just my assumption. I care about the referrer's API but others might not. It seems reasonable to make the toggle most apparent for the more-likely-to-break change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
😸 Shall we rename
storage.oci.distribution-methodto something more format-centric, likestorage.oci.bundle-formatwith values legacy (DSSE + tag-based) and sigstore-bundle (protobuf bundle + referrers API). The referrers API becomes an implementation consequence of choosing the bundle format, rather than the primary toggle. This also aligns better with how users will reason about cosign v4 compatibility.Would that naming satisfy your concern? Happy to do the rename in this PR if it gets us to approval.
we could extend a similar configuration for each non-oci backend as we progress with the changes.
storage.gcs.bundle-format, etc.,There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you were to name as
bundle-format, I assume that would just be atrue/falseconfiguration? If you want the configuration to be clearly legacy vs bundle, astorage-formatorencoding-formatmight make more sense? Instead of havinglegacy/bundleoptions, it might also be clearer to have it beDSSE/bundleas that is the actual format?So maybe the suggestion would be
storage.oci.formatwith available options ofDSSE(default for now) andbundle?