Skip to content

Add KEP-4680 device health reporting with message field support#127

Draft
harche wants to merge 1 commit into
kubernetes-sigs:mainfrom
harche:healt_status
Draft

Add KEP-4680 device health reporting with message field support#127
harche wants to merge 1 commit into
kubernetes-sigs:mainfrom
harche:healt_status

Conversation

@harche

@harche harche commented Nov 11, 2025

Copy link
Copy Markdown
Contributor

Summary

Adds KEP-4680 device health reporting to the example driver. The driver implements the DRAResourceHealthServer gRPC interface (dra-health/v1alpha1), allowing kubelet to stream device health updates that appear in pod.status.containerStatuses[].allocatedResourcesStatus.

  • Health monitoring is always enabled (Beta feature, default-on in K8s 1.36)
  • A health simulator generates realistic scenarios (temperature warnings, ECC errors, communication failures, recovery cycles)
  • Uses actual device names from the allocatable set (works with partitioned devices)

Verification

kubectl get pod -n health-reporting pod0 \
  -o jsonpath='{.status.containerStatuses[0].allocatedResourcesStatus}' | jq .
[
  {
    "name": "claim:gpu",
    "resources": [
      {
        "health": "Healthy",
        "message": "Device gpu-0 operating normally, temperature: 46°C",
        "resourceID": "k8s.gpu.example.com/gpu=common"
      }
    ]
  }
]

Cluster requirements

  • Kubernetes 1.36+ (ResourceHealthStatus and ResourceHealthStatusMessage feature gates are Beta and enabled by default)

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Nov 11, 2025
@k8s-ci-robot

Copy link
Copy Markdown
Contributor

Welcome @harche!

It looks like this is your first PR to kubernetes-sigs/dra-example-driver 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/dra-example-driver has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: harche
Once this PR has been reviewed and has the lgtm label, please assign klueska for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot requested review from byako and pohly November 11, 2025 17:23
@k8s-ci-robot k8s-ci-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Nov 11, 2025
@harche

harche commented Nov 11, 2025

Copy link
Copy Markdown
Contributor Author

/hold for the implementation PRs to merge,

kubernetes/enhancements#4680 (comment)

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 11, 2025
@nojnhuh nojnhuh moved this from 🆕 New to 🏗 In progress in Dynamic Resource Allocation Nov 13, 2025
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 10, 2026
@pohly pohly moved this from 🏗 In progress to 👀 In review in Dynamic Resource Allocation Feb 20, 2026
@pohly pohly moved this from 👀 In review to 🏗 In progress in Dynamic Resource Allocation Mar 2, 2026
@harche

harche commented Mar 13, 2026

Copy link
Copy Markdown
Contributor Author

I will update once 1.36 beta is cut

@pohly pohly moved this from 🏗 In progress to 👀 In review in Dynamic Resource Allocation Mar 16, 2026
@pohly pohly moved this from 👀 In review to 🏗 In progress in Dynamic Resource Allocation Mar 16, 2026
@pohly pohly moved this from 🏗 In progress to 👀 In review in Dynamic Resource Allocation Apr 7, 2026
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 13, 2026
@harche

harche commented May 13, 2026

Copy link
Copy Markdown
Contributor Author

/cc @SergeyKanzhelev @nojnhuh

@harche

harche commented May 13, 2026

Copy link
Copy Markdown
Contributor Author

I will update once 1.36 beta is cut

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 13, 2026
@harche

harche commented May 13, 2026

Copy link
Copy Markdown
Contributor Author

/cc @SergeyKanzhelev @nojnhuh

/hold I am adding a feature to trigger the health status update in this example driver.

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 13, 2026
@harche harche marked this pull request as draft May 14, 2026 16:13
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 14, 2026
@harche

harche commented May 14, 2026

Copy link
Copy Markdown
Contributor Author

Putting this on hold until #201 merges. Both PRs modify the e2e test infrastructure (e2e_setup_test.go, e2e_test.go) and will conflict. Will rebase after #201 lands.

/hold

@pohly

pohly commented May 15, 2026

Copy link
Copy Markdown
Contributor

As mentioned before, I don't like that DRA drivers have to interact with the gRPC interface (drahealthv1alpha1 currently, despite the beta graduation of the feature?!). Can we first improve the kubeletplugin support for this feature (abstract from underlying gRPC interface similar to how it's done for the main DRA kubelet interace, better documentation) and only then make the DRA example driver use that improved support?

@pohly pohly moved this from 👀 In review to 📋 Backlog in Dynamic Resource Allocation Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants