Skip to content

Add DRA status feedback to hub-and-spoke ManifestWork#1293

Merged
kubernetes-prow[bot] merged 1 commit into
kubernetes-sigs:mainfrom
TomerNewman:MGMT-24568-dra-manifestwork-status
Jun 25, 2026
Merged

Add DRA status feedback to hub-and-spoke ManifestWork#1293
kubernetes-prow[bot] merged 1 commit into
kubernetes-sigs:mainfrom
TomerNewman:MGMT-24568-dra-manifestwork-status

Conversation

@TomerNewman

@TomerNewman TomerNewman commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add three DRA status feedback JSON path entries (dra.availableNumber, dra.desiredNumber, dra.nodesMatchingSelectorNumber) to the ManifestWork feedback rules, mirroring the existing devicePlugin.* pattern so DRA status is reported back from spoke to hub.
  • Add DRA hostPath volume validation to the webhook, enforcing the same allowlist (/dev, /sys, /var, /opt) used for DevicePlugin volumes.

Changes

internal/manifestwork/manifestwork.go

Added 3 JsonPath entries for DRA status to moduleStatusJSONPaths, maintaining alphabetical grouping between devicePlugin.* and moduleLoader.*.

internal/manifestwork/manifestwork_test.go

Added a Describe("moduleStatusJSONPaths") block using ConsistOf to assert all 9 expected feedback paths (3 devicePlugin + 3 DRA + 3 moduleLoader).

internal/webhook/module.go

Added validateDRAVolumes() function and wired it into validateModule(). This validates spec.dra.volumes hostPath entries against the same allowlist used for DevicePlugin, with path traversal protection via filepath.Clean.

internal/webhook/module_test.go

Added Describe("validateDRAVolumes") with 6 test cases covering: nil DRA, no volumes, non-hostPath, allowed hostPath, disallowed hostPath, and path traversal.

Test plan

  • Unit tests pass (go test ./internal/manifestwork/ ./internal/webhook/)
  • Hub-and-spoke e2e verified on minikube (K8s 1.35):
    • ManifestWork created with all 9 feedback paths (3 DP + 3 DRA + 3 ML)
    • Module propagated to spoke with DRA spec intact (driverName, container, volumes)
    • DRA DaemonSet created on spoke with correct labels, volumes, nodeSelector
    • DRA status feedback (dra.desiredNumber, dra.nodesMatchingSelectorNumber) flowing back to hub
    • Hub webhook rejects disallowed DRA hostPath volumes (/etc/shadow)
    • Hub webhook rejects path traversal (/dev/../etc/shadow)
    • Hub webhook accepts allowed DRA hostPaths (/dev/vfio, /sys/bus/pci)

/cc @yevgeny-shnaidman @ybettan
/assign @ybettan @yevgeny-shnaidman
/uncc @mresvanis

@netlify

netlify Bot commented Jun 23, 2026

Copy link
Copy Markdown

Deploy Preview for kubernetes-sigs-kmm ready!

Name Link
🔨 Latest commit 4dcedf3
🔍 Latest deploy log https://app.netlify.com/projects/kubernetes-sigs-kmm/deploys/6a3cd83b5fe527000810662e
😎 Deploy Preview https://deploy-preview-1293--kubernetes-sigs-kmm.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@kubernetes-prow kubernetes-prow Bot requested a review from mresvanis June 23, 2026 10:35
@kubernetes-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: TomerNewman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubernetes-prow kubernetes-prow Bot requested a review from ybettan June 23, 2026 10:35
@kubernetes-prow kubernetes-prow Bot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. approved Indicates a PR has been approved by an approver from all required OWNERS files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 23, 2026
@TomerNewman

Copy link
Copy Markdown
Collaborator Author

@kubernetes-prow kubernetes-prow Bot requested review from yevgeny-shnaidman and removed request for mresvanis June 23, 2026 10:36
@codecov-commenter

codecov-commenter commented Jun 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 33.33333% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.67%. Comparing base (fa23a9b) to head (4dcedf3).
⚠️ Report is 388 commits behind head on main.

Files with missing lines Patch % Lines
internal/webhook/module.go 33.33% 4 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1293      +/-   ##
==========================================
- Coverage   79.09%   73.67%   -5.43%     
==========================================
  Files          51       67      +16     
  Lines        5109     5056      -53     
==========================================
- Hits         4041     3725     -316     
- Misses        882     1158     +276     
+ Partials      186      173      -13     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread internal/webhook/module.go Outdated
Comment on lines +263 to +278
func validateDRAVolumes(dra *kmmv1beta1.DRASpec) error {
if dra == nil {
return nil
}

for i, vol := range dra.Volumes {
if vol.HostPath != nil && !isAllowedHostPath(vol.HostPath.Path) {
return fmt.Errorf(
"spec.dra.volumes[%d]: hostPath %q is not allowed; only /dev, /sys, /var and /opt paths are permitted",
i, vol.HostPath.Path,
)
}
}

return nil
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about creating a shared volume checker function to be used in both devicePlugin and DRA.

The function can look something like:

func validateHostPathVolumes(fieldPath string, volumes []v1.Volume) error {
      for i, vol := range volumes {
              if vol.HostPath != nil && !isAllowedHostPath(vol.HostPath.Path) {
                      return fmt.Errorf(
                              "%s.volumes[%d]: hostPath %q is not allowed; only /dev, /sys, /var and /opt paths are permitted",
                              fieldPath, i, vol.HostPath.Path,
                      )
              }
      }
      return nil
}

and then the callers boils down to

// in validateModule():

if mod.Spec.DRA != nil {
    if err := validateHostPathVolumes("spec.dra", mod.Spec.DRA.Volumes); err != nil {
        return nil, fmt.Errorf("failed to validate DRA volumes: %v", err)
    }
}

if mod.Spec.DevicePlugin != nil {
    if err := validateHostPathVolumes("spec.devicePlugin", mod.Spec.DevicePlugin.Volumes); err != nil {
        return nil, fmt.Errorf("failed to validate device plugin volumes: %v", err)
    }
}

Add three DRA status feedback JSON path entries
(dra.availableNumber, dra.desiredNumber,
dra.nodesMatchingSelectorNumber) to the ManifestWork
feedback rules, mirroring the existing devicePlugin
pattern.

Validate DRA hostPath volumes against the same allowlist
(/dev, /sys, /var, /opt) used for DevicePlugin volumes,
ensuring the hub webhook rejects disallowed mounts before
they reach spoke clusters.
@TomerNewman TomerNewman force-pushed the MGMT-24568-dra-manifestwork-status branch from 412d715 to 4dcedf3 Compare June 25, 2026 07:26
@ybettan

ybettan commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

/lgtm

@kubernetes-prow kubernetes-prow Bot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 25, 2026
@TomerNewman

Copy link
Copy Markdown
Collaborator Author

/retest

1 similar comment
@TomerNewman

Copy link
Copy Markdown
Collaborator Author

/retest

@kubernetes-prow kubernetes-prow Bot merged commit ab43e05 into kubernetes-sigs:main Jun 25, 2026
23 checks passed
@TomerNewman TomerNewman deleted the MGMT-24568-dra-manifestwork-status branch June 25, 2026 10:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants