Skip to content

OCPBUGS-83580: Load fuse kernel module before testing /dev/fuse in CRI-O #31044

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Chandan9112 wants to merge 1 commit into
base: main
Choose a base branch
from
+11 −1
Open

OCPBUGS-83580: Load fuse kernel module before testing /dev/fuse in CRI-O #31044

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion test/extended/node/node_e2e/node.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,8 +112,18 @@ var _ = g.Describe("[sig-node] [Jira:Node/Kubelet] Kubelet, CRI-O, CPU manager",
podName := "pod-devfuse"
ns := "devfuse-test"

g.By("Check if the default CRI-O runtime is runc")
workerName, err := oc.AsAdmin().WithoutNamespace().Run("get").Args("nodes", "-l", "node-role.kubernetes.io/worker", "-o=jsonpath={.items[0].metadata.name}").Output()
o.Expect(err).NotTo(o.HaveOccurred())
o.Expect(workerName).NotTo(o.BeEmpty(), "No worker nodes found")
runtime, err := nodeutils.ExecOnNodeWithChroot(oc, strings.TrimSpace(workerName), "/bin/bash", "-c", "crio config 2>/dev/null | grep 'default_runtime'")
o.Expect(err).NotTo(o.HaveOccurred())
if strings.Contains(runtime, "runc") {
g.Skip("Skipping: not applicable to runc runtime")
}
Comment on lines +115 to +123
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Apr 27, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

The runtime skip doesn’t replace the fuse-module precondition.

This still goes straight to pod setup without any host-side modprobe fuse check, so the original /dev/fuse flake can still happen on non-runc workers. Please keep the fuse-module step before this logic, or verify /dev/fuse exists on the worker before continuing.

Suggested fix 
 g.By("Check if the default CRI-O runtime is runc")
 workerName, err := oc.AsAdmin().WithoutNamespace().Run("get").Args("nodes", "-l", "node-role.kubernetes.io/worker", "-o=jsonpath={.items[0].metadata.name}").Output()
 o.Expect(err).NotTo(o.HaveOccurred())
 o.Expect(workerName).NotTo(o.BeEmpty(), "No worker nodes found")
 runtime, err := nodeutils.ExecOnNodeWithChroot(oc, strings.TrimSpace(workerName), "/bin/bash", "-c", "crio config 2>/dev/null | grep 'default_runtime'")
 o.Expect(err).NotTo(o.HaveOccurred())
 if strings.Contains(runtime, "runc") {
 	g.Skip("Skipping: not applicable to runc runtime")
 }
+
+g.By("Ensure fuse kernel module is loaded before creating the pod")
+_, err = nodeutils.ExecOnNodeWithChroot(oc, strings.TrimSpace(workerName), "/bin/bash", "-c", "modprobe fuse")
+o.Expect(err).NotTo(o.HaveOccurred())
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
g.By("Check if the default CRI-O runtime is runc")
workerName, err := oc.AsAdmin().WithoutNamespace().Run("get").Args("nodes", "-l", "node-role.kubernetes.io/worker", "-o=jsonpath={.items[0].metadata.name}").Output()
o.Expect(err).NotTo(o.HaveOccurred())
o.Expect(workerName).NotTo(o.BeEmpty(), "No worker nodes found")
runtime, err := nodeutils.ExecOnNodeWithChroot(oc, strings.TrimSpace(workerName), "/bin/bash", "-c", "crio config 2>/dev/null | grep 'default_runtime'")
o.Expect(err).NotTo(o.HaveOccurred())
if strings.Contains(runtime, "runc") {
g.Skip("Skipping: not applicable to runc runtime")
}
g.By("Check if the default CRI-O runtime is runc")
workerName, err := oc.AsAdmin().WithoutNamespace().Run("get").Args("nodes", "-l", "node-role.kubernetes.io/worker", "-o=jsonpath={.items[0].metadata.name}").Output()
o.Expect(err).NotTo(o.HaveOccurred())
o.Expect(workerName).NotTo(o.BeEmpty(), "No worker nodes found")
runtime, err := nodeutils.ExecOnNodeWithChroot(oc, strings.TrimSpace(workerName), "/bin/bash", "-c", "crio config 2>/dev/null | grep 'default_runtime'")
o.Expect(err).NotTo(o.HaveOccurred())
if strings.Contains(runtime, "runc") {
g.Skip("Skipping: not applicable to runc runtime")
}
g.By("Ensure fuse kernel module is loaded before creating the pod")
_, err = nodeutils.ExecOnNodeWithChroot(oc, strings.TrimSpace(workerName), "/bin/bash", "-c", "modprobe fuse")
o.Expect(err).NotTo(o.HaveOccurred())
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@test/extended/node/node_e2e/node.go` around lines 115 - 123, The
runtime-based skip removed the prior host-side fuse-module precondition and can
allow tests to proceed without /dev/fuse present; restore the fuse check or add
an explicit /dev/fuse verification on the worker before continuing.
Specifically, before the runtime check that uses workerName and
nodeutils.ExecOnNodeWithChroot, either reinsert the modprobe/ensure-fuse step
that was originally run on the host or run a command via
nodeutils.ExecOnNodeWithChroot (using strings.TrimSpace(workerName)) to test for
the presence of /dev/fuse (e.g., test -e /dev/fuse) and fail/skip if missing;
ensure this check executes unconditionally prior to creating pods so the
original fuse-module precondition is enforced.

Copy link
Copy Markdown
Contributor Author

@Chandan9112 Chandan9112 Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All failures were on clusters using runc as the default CRI-O runtime, which does not include io.kubernetes.cri-o.Devices in its allowed_annotations. On crun clusters, /dev/fuse has always been available and the test has never failed.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Chandan9112, thank you for the detailed clarification. That makes sense — since io.kubernetes.cri-o.Devices is not listed in allowed_annotations for runc-based clusters, CRI-O would silently ignore the device annotation regardless of whether /dev/fuse exists on the host, making the test meaningless and prone to false positives/failures in that environment. Skipping early on runc is the correct fix. I'll withdraw my concern.

📝 Learnings were identified and not saved (knowledge base disabled). Enable


g.By("Create a test namespace")
err := oc.AsAdmin().WithoutNamespace().Run("create").Args("namespace", ns).Execute()
err = oc.AsAdmin().WithoutNamespace().Run("create").Args("namespace", ns).Execute()
o.Expect(err).NotTo(o.HaveOccurred())
defer oc.AsAdmin().WithoutNamespace().Run("delete").Args("namespace", ns, "--ignore-not-found").Execute()

Expand Down