Auto RDMA device injection for GPU containers on Kubernetes (no privileged mode)

[jiusanzhou]

Hey DeepSpeed community! 👋

Running DeepSpeed distributed training on Kubernetes with InfiniBand/RoCE? You've probably hit RDMA device management pain:

• Containers need /dev/infiniband/* for NCCL RDMA transport, but don't have access by default
• Common fix: privileged: true or manual device mounts — fragile and insecure
• On multi-NIC nodes, NCCL may pick the wrong RDMA card (cross-NUMA), degrading all-reduce performance

I built k8s-rdma-device-plugin to solve this:

1. Auto-injects RDMA device permissions via containerd NRI — no privileged: true
2. GPU-RDMA PCIe affinity — discovers which RDMA NIC is closest to each GPU and injects only the right devices
3. Optional resource accounting — reports rdma.io/hca to kubelet

With gpuRdmaAutoInject=true, any pod with NVIDIA_VISIBLE_DEVICES automatically gets the correct RDMA devices injected based on PCIe topology. Zero manual configuration needed.

Quick deploy

helm install rdma-device-plugin ./deploy/charts \
  --namespace kube-system \
  --set gpuRdmaAutoInject=true

Repo: https://github.com/jiusanzhou/k8s-rdma-device-plugin

Feedback and PRs welcome!