Skip to content

Allow some Sonobuoy tests (KaaS) to fail. - #1092

Merged
mbuechse merged 1 commit into
mainfrom
allow-some-sonobuoy-tests-to-fail
Feb 17, 2026
Merged

Allow some Sonobuoy tests (KaaS) to fail.#1092
mbuechse merged 1 commit into
mainfrom
allow-some-sonobuoy-tests-to-fail

Conversation

@guettli

@guettli guettli commented Feb 16, 2026

Copy link
Copy Markdown
Contributor

Closes #1087

Comment thread Tests/kaas/sonobuoy_handler/run_sonobuoy.py Outdated
Comment thread Tests/kaas/sonobuoy_handler/run_sonobuoy.py
Comment thread Tests/kaas/sonobuoy_handler/sonobuoy_handler.py Outdated
Comment thread Tests/kaas/sonobuoy_handler/sonobuoy_handler.py Outdated
Comment thread Tests/kaas/sonobuoy_handler/sonobuoy_handler.py Outdated
Comment thread Tests/kaas/sonobuoy_handler/sonobuoy_handler.py Outdated
Comment on lines +157 to +158
if not isinstance(config_obj, dict):
raise ValueError(f"Invalid sonobuoy config format in {sonobuoy_config}: top-level YAML object must be a mapping")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would omit all the syntax checks for the YAML. If we want that, we can use some kind of YAML schema validator or something. I would do minimal checks, but I would catch any exception that occurs while loading the file saying there might be a syntax error. We don't expect that to happen often because we control the YAML file ourselves, and we don't expect it to change very often. Most likely change is that we remove entries once the issues with Cilium go away one by one.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mbuechse I would like the code to fail, if the config contains typos. I would like the command to fail, if someone writes "regexs" instead of "regex". But if it is very important for you, I will remove the syntax checks.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose the code will fail with a KeyError or NameError, which is not as nice. I think using a schema validator (if we deem it necessary) would spare us from a lot of boilerplate code here. But I won't die on a hill fighting over this...

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then I would like to leave the code as it is. I really do not like it, when a typo in a config get silently ignored.

Comment thread Tests/kaas/sonobuoy_handler/sonobuoy_handler.py Outdated
@guettli
guettli force-pushed the allow-some-sonobuoy-tests-to-fail branch from f8a482c to 4e19052 Compare February 17, 2026 11:23
@guettli
guettli requested a review from mbuechse February 17, 2026 11:24
@guettli
guettli force-pushed the allow-some-sonobuoy-tests-to-fail branch from 4e19052 to 9d88f8b Compare February 17, 2026 12:03
Signed-off-by: Thomas Güttler <thomas.guettler@syself.com>
@guettli
guettli force-pushed the allow-some-sonobuoy-tests-to-fail branch from 9d88f8b to f29b4c4 Compare February 17, 2026 12:05

@mbuechse mbuechse left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is still potential for improvement, but let's see how this works.

@mbuechse
mbuechse merged commit 1bebabc into main Feb 17, 2026
9 checks passed
@mbuechse
mbuechse deleted the allow-some-sonobuoy-tests-to-fail branch February 17, 2026 12:18
@guettli

guettli commented Feb 17, 2026

Copy link
Copy Markdown
Contributor Author

@mbuechse thank you for your support. Our Syself Kubernetes 1.34 Clusters seem to be pass all tests now.

I will update the logs and the report.

BTW, in the long run many failed_ok regex are no longer needed because sig-network will accept our PR

report-2026-02-17.yaml.txt

syself-2026-02-17.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Filter some failed KaaS Tests

2 participants