Unable to deploy pod that uses gpu-device-plugin, `failed to generate spec: failed to mkdir`

[UntouchedWagons]

Describe the bug

Kubernetes pods that want to use an intel i915 gpu fail to be created, the following error is provided by Kubernetes:

Warning Failed 6s (x13 over 2m40s) kubelet spec.containers{vainfo}: (combined from similar events): Error: failed to generate container "79bcfdf507c2346c290f11ea5ac0381704751ce3f902d8b082fd75d8f791163f" spec: failed to generate spec: failed to mkdir "/dev/dri/by-path/pci-0000:00:02.0-platform-simple-framebuffer.0-card": mkdir /dev/dri/by-path/pci-0000:00:02.0-platform-simple-framebuffer.0-card: file exists

To Reproduce

Deploy single-node cluster on Ubuntu 26.04 with Intel GPU installed. Install Cert-manager, node feature discovery, intel-device-plugins-operator and intel-device-plugins-gpu helm charts then apply this vainfo pod.

Expected behavior

The vainfo pod to start and list supported encode/decode profiles in its stdout.

Screenshots

Not sure what could be added

System (please complete the following information):

• OS version: Ubuntu 26.04 Server
• Kernel version: 7.0.0-22-generic
• Device plugins version: intel-device-plugins-operator (0.36.0), intel-device-plugins-gpu (0.36.0)
• Hardware info: MinisForum MS-01 with Intel Core i5-12600H

Additional context

This identical deployment works perfectly on Debian 13.

[eero-t]

Some workloads need by-path symlinks in the container to work, but handling them has been always a bit of a pain due to OS & other differences (see e.g. #2158, #1590, #1564).

OS version: Ubuntu 26.04 Server
...
This identical deployment works perfectly on Debian 13.

Which container engines (runc, crun, docker, podman, containerd, CRI-O...) you're using on them i.e. which programs / daemons are responsible for handling the CDI/OCI spec(s)?

PS. I'm not developer in this project, just (very) occasional contributor...

[tkatila]

Hi @UntouchedWagons, can you try setting bypathMode as none ?

I wonder why the by-path entities are there already..

[UntouchedWagons]

Which container engines (runc, crun, docker, podman, containerd, CRI-O...) you're using on them i.e. which programs / daemons are responsible for handling the CDI/OCI spec(s)?

I'm using whatever comes with k3s by default, I think containerd?

Hi @UntouchedWagons, can you try setting bypathMode as none ?

Will do.

[eero-t]

Which container engines (runc, crun, docker, podman, containerd, CRI-O...) you're using on them i.e. which programs / daemons are responsible for handling the CDI/OCI spec(s)?

I'm using whatever comes with k3s by default, I think containerd?

Seems to be containerd + runc: https://github.com/k3s-io/k3s/releases

Which K3s version you're using?

This identical deployment works perfectly on Debian 13.

With same K3s version?

[UntouchedWagons]

With same K3s version?

Yes, v1.36.1+k3s1

Hi @UntouchedWagons, can you try setting bypathMode as none ?

Setting bypathMode to none worked.