Skip to content

fix: opt into fixed klog stderrthreshold behavior#3944

Open
pierluigilenoci wants to merge 2 commits intokcp-dev:mainfrom
pierluigilenoci:fix/honor-stderrthreshold
Open

fix: opt into fixed klog stderrthreshold behavior#3944
pierluigilenoci wants to merge 2 commits intokcp-dev:mainfrom
pierluigilenoci:fix/honor-stderrthreshold

Conversation

@pierluigilenoci
Copy link
Copy Markdown

@pierluigilenoci pierluigilenoci commented Mar 26, 2026

Problem

klog v2 defaults -logtostderr to true. When this flag is active, the -stderrthreshold flag is silently ignored — all log messages of every severity (INFO, WARNING, ERROR, FATAL) are unconditionally written to stderr.

This has been an open issue since 2020: kubernetes/klog#212

Fix

Bump klog from v2.130.1 to v2.140.0 in both staging modules (cli, code-generator) and opt into the fixed behavior in all 6 binaries:

  • CLI tools: kubectl-kcp, kubectl-ws, kubectl-create-workspace
  • Code generators: cluster-client-gen, cluster-informer-gen, cluster-lister-gen

The default stderrthreshold remains INFO so the observable behavior is unchanged.

References

Opt into fixed klog stderrthreshold behavior so that -stderrthreshold is honored even when -logtostderr=true.

@kcp-ci-bot kcp-ci-bot requested review from ncdc and sttts March 26, 2026 23:07
@kcp-ci-bot kcp-ci-bot added dco-signoff: yes Indicates the PR's author has signed the DCO. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Mar 26, 2026
@kcp-ci-bot
Copy link
Copy Markdown
Contributor

Hi @pierluigilenoci. Thanks for your PR.

I'm waiting for a kcp-dev member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@kcp-ci-bot kcp-ci-bot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Mar 26, 2026
Copy link
Copy Markdown
Member

@ntnn ntnn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/ok-to-test

@kcp-ci-bot kcp-ci-bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Mar 27, 2026
@ntnn ntnn added this to tbd Mar 27, 2026
@github-project-automation github-project-automation Bot moved this to Backlog in tbd Mar 27, 2026
@ntnn ntnn moved this from Backlog to Reviewing in tbd Mar 28, 2026
@ntnn
Copy link
Copy Markdown
Member

ntnn commented Mar 28, 2026

/retest

That's odd. Might be the infra flaking.

@pierluigilenoci
Copy link
Copy Markdown
Author

/retest

@pierluigilenoci
Copy link
Copy Markdown
Author

/kind bug

@kcp-ci-bot kcp-ci-bot added kind/bug Categorizes issue or PR as related to a bug. and removed do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. labels Mar 28, 2026
@mjudeikis
Copy link
Copy Markdown
Contributor

I would want to wait until the same version of klog lands here via rebase. Now this unaligns us from upstream
/hold

@kcp-ci-bot kcp-ci-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 30, 2026
@pierluigilenoci
Copy link
Copy Markdown
Author

@mjudeikis Thanks for the review and the context — that makes sense.

You're right that this PR bumps k8s.io/klog/v2 from v2.130.1 to v2.140.0 across all go.mod files, which would misalign from upstream until the next rebase. I understand wanting to avoid that.

Happy to keep this on hold until klog v2.140.0 (or newer) lands via upstream rebase. Once it does, the go.mod changes can be dropped and only the flag.Set calls would remain. No rush on our side — just ping me when the rebase happens and I'll rebase/update accordingly.

@kcp-ci-bot kcp-ci-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 31, 2026
@pierluigilenoci
Copy link
Copy Markdown
Author

Hi @mjudeikis — just a heads-up: klog v2.140.0 was released on March 6, 2026 (https://github.com/kubernetes/klog/releases/tag/v2.140.0), so the upstream dependency is now available.

I notice this PR also has a merge conflict that needs resolving. Would you like me to rebase and update the PR to align with the current main branch? Once rebased, the /hold could be removed since the klog version you were waiting for is now published.

Let me know how you'd like to proceed. Thank you!

@pierluigilenoci pierluigilenoci deleted the fix/honor-stderrthreshold branch April 10, 2026 14:55
@github-project-automation github-project-automation Bot moved this from Reviewing to Done in tbd Apr 10, 2026
@pierluigilenoci pierluigilenoci restored the fix/honor-stderrthreshold branch April 10, 2026 14:56
@pierluigilenoci pierluigilenoci force-pushed the fix/honor-stderrthreshold branch from a7eca18 to c3238ca Compare April 10, 2026 14:59
@kcp-ci-bot kcp-ci-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 10, 2026
@ntnn
Copy link
Copy Markdown
Member

ntnn commented Apr 10, 2026

Hey, we are waiting to rebase our kubernetes fork on a kubernetes version, which will likely be released at the end of this month. When that is released we'll rebase our fork and kcp and then this pr can be rebased.

@ntnn ntnn moved this from Done to Reviewing in tbd Apr 10, 2026
@pierluigilenoci
Copy link
Copy Markdown
Author

CI fix pushed

Root cause: The virtual-workspace-framework staging module was missed when bumping k8s.io/klog/v2 from v2.130.1 to v2.140.0 across the monorepo. This caused three CI failures:

Check Failure Root cause
pull-kcp-lint "no go files to analyze" in virtual-workspace-framework stale go.mod caused module resolution issues
pull-kcp-verify "go.mod and/or go.sum have been modified" go mod tidy upgraded klog, creating uncommitted changes
pull-kcp-test-unit "no Go files" / "go mod tidy needed" same stale module issue

Fix: bumped k8s.io/klog/v2 to v2.140.0 in staging/src/github.com/kcp-dev/virtual-workspace-framework/go.mod and ran go mod tidy.

Regarding pull-kcp-test-e2e-multiple-runs: TestUseWorkspace failed with "the server could not find the requested resource (post workspaces.tenancy.kcp.io)" — this is an infra timing flake unrelated to the klog changes. The new push should re-trigger all checks.

@pierluigilenoci
Copy link
Copy Markdown
Author

/retest

@ntnn ntnn mentioned this pull request Apr 21, 2026
1 task
@pierluigilenoci
Copy link
Copy Markdown
Author

Hi @ntnn @mjudeikis — I noticed the K8s fork bump landed on Apr 20 (24a0254), but klog remains at v2.130.1 in main. Since klog v2.140.0 didn't come through the rebase, would it make sense to lift the /hold and bump klog separately in this PR? Happy to rebase and adjust as needed. Thanks!

@kcp-ci-bot kcp-ci-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 21, 2026
@ntnn
Copy link
Copy Markdown
Member

ntnn commented Apr 21, 2026

Hi; that was just a bump of the old rebase to handle an unrelated ticket - namely making the upstream kubernetes garbage collector cluster-aware.
The rebase is actively being worked on and I linked this PR in the ticket for the rebase so it can be handled alongside it.

@pierluigilenoci pierluigilenoci force-pushed the fix/honor-stderrthreshold branch from 62ee75f to c187e3a Compare April 21, 2026 22:27
@kcp-ci-bot kcp-ci-bot added do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. release-note Denotes a PR that will be considered when it comes time to generate release notes. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Apr 21, 2026
klog v2 defaults -logtostderr to true. When this flag is active, the
-stderrthreshold flag is silently ignored — all log messages are
unconditionally written to stderr regardless of severity.

Bump klog from v2.130.1 to v2.140.0 in both staging modules (cli,
code-generator) and opt into the fixed behavior in all 6 binaries
(kubectl-kcp, kubectl-ws, kubectl-create-workspace, cluster-client-gen,
cluster-informer-gen, cluster-lister-gen).

The default stderrthreshold remains INFO so the observable behavior is
unchanged.

Ref: kubernetes/klog#212, kubernetes/klog#432
Signed-off-by: Pierluigi Lenoci <pierluigilenoci@gmail.com>
@pierluigilenoci pierluigilenoci force-pushed the fix/honor-stderrthreshold branch from c187e3a to be2a86b Compare May 2, 2026 09:44
@kcp-ci-bot kcp-ci-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 2, 2026
Align all go.mod files in staging/ to use the same klog version
as the root module, fixing verify-go-modules CI check.

Signed-off-by: Pierluigi Lenoci <pierluigilenoci@gmail.com>
@kcp-ci-bot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign mjudeikis for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kcp-ci-bot
Copy link
Copy Markdown
Contributor

kcp-ci-bot commented May 3, 2026

@pierluigilenoci: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kcp-verify 65b2a4c link true /test pull-kcp-verify
pull-kcp-verify-codegen 65b2a4c link true /test pull-kcp-verify-codegen

Full PR test history

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the DCO. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. kind/bug Categorizes issue or PR as related to a bug. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants