Skip to content

fix: use distro-based images for credential plugin integration#191

Open
porridge wants to merge 7 commits intomasterfrom
distro
Open

fix: use distro-based images for credential plugin integration#191
porridge wants to merge 7 commits intomasterfrom
distro

Conversation

@porridge
Copy link
Copy Markdown
Collaborator

@porridge porridge commented May 6, 2026

This PR is on top of #190, you might want to look only at the latest commits.

Why

  • Turns out that feat: initial image cred plugin integration #184 never actually worked in practice, since the FROM scratch image does not contain libraries that the GKE credential plugin needs: plugin execution failed: fork/exec /tmp/credential-provider-bin/auth-provider-gcp: no such file or directory, stderr:
  • Looks like the testing done for the above PR in fix(ci): work around preloaded image use problem in recent k8s stackrox#19287 where the gke-latest-... job succeeded must have been some kind of fluke 🤷🏻 In fact that problematic test case didn't seem to run at all:
    15:10:32 | INFO  | ImageScanningTest         | Starting testcase: Image metadata from registry test - quay-auto
    15:10:32 | INFO  | Helpers                   | Won't collect logs for: org.opentest4j.TestAbortedException: Ignored via @IgnoreIf
    15:10:32 | INFO  | ImageScanningTest         | Post test cleanup:
    15:10:32 | INFO  | ImageScanningTest         | Ending testcase
    

What this does

  • This PR adds a parallel set of images differing only in base image, they are about twice the size of the FROM scratch ones. It also changes the deploy tool to use them transparently when credential plugin integration is enabled.
    image

  • Additionally, it changes the auth data passing to resemble the way kubelet does it more - in particular it drops the server endpoint piece which somehow prevents CRI from fetching the image:

    msg="image failed to pull" image=us.gcr.io/acs-san-stackroxci/qa-multi-arch:nginx-1.12 authNum=3
    error="rpc error: code = Unknown desc = failed to pull and unpack image \"us.gcr.io/acs-san-stackroxci/qa-multi-arch:nginx-1.12\": failed to resolve reference \"us.gcr.io/acs-san-stackroxci/qa-multi-arch:nginx-1.12\": failed to authorize: failed to fetch anonymous token: unexpected status from GET request to https://us.gcr.io/v2/token?scope=repository%3Aacs-san-stackroxci%2Fqa-multi-arch%3Apull&scope=repository%3Aacs-san-stackroxci%2Fus.gcr.io%2Fqa-multi-arch%3Apull&service=us.gcr.io: 403 Forbidden"
    

Testing

time=2026-05-06T13:11:41.995Z level=INFO source=/home/runner/work/image-prefetcher/image-prefetcher/internal/main.go:187 msg="image pulled successfully" image=us.gcr.io/acs-san-stackroxci/qa-multi-arch:nginx-1.12 authNum=0 authServer="" authUsername=_token response="image_ref:\"sha256:4037a5562b030fd80ec889bb885405587a52cfef898ffb7402649005dfda75ff\"" elapsed=28.646953919s
time=2026-05-06T13:11:50.222Z level=INFO source=/home/runner/work/image-prefetcher/image-prefetcher/internal/main.go:187 msg="image pulled successfully" image=us.gcr.io/acs-san-stackroxci/qa/registry-image:0.3 authNum=1 authServer="" authUsername=_token response="image_ref:\"sha256:e03ee8c409b34496c09c261194dd3d0d825f0a67350d49c8812d7dd65a95dfdc\"" elapsed=36.874397134s

porridge and others added 5 commits May 6, 2026 11:48
Replace the scheme-based decoding of CredentialProviderConfig (which
required k8s.io/kubernetes/pkg/kubelet/apis/config) with minimal local
struct definitions and sigs.k8s.io/yaml unmarshalling. This removes
k8s.io/kubernetes and its transitive dependencies from go.mod.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The credential provider binary needs a real OS (not scratch), so
select the -distro image variant automatically.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@porridge porridge changed the title Distro fix: use distro-based images for credential plugin integration May 6, 2026
porridge added 2 commits May 6, 2026 14:27
Signed-Off-By: Marcin Owsiany <porridge@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant