[Utils] Add ci_results script#1303
Merged
Merged
Conversation
Contributor
Author
|
Some example output for $ py utils/ci_status.py current-status windows-nvidia-clang-d3d12.yaml windows-amd-dxc-warp-d3d12.yaml
## windows-nvidia-clang-d3d12.yaml (success)
- Name: Windows D3D12 NVIDIA Clang
- Timestamp: 2026-06-11T00:37:07Z
- llvm/llvm-project: 3c7cea8273bd06fdd9f84197293fb3d7f4b94435
- llvm/offload-test-suite: 0d3cc83a8c3570c11026aac1440c50726a671e5a
## windows-amd-dxc-warp-d3d12.yaml (failure)
- Name: Windows D3D12 Warp DXC
- Timestamp: 2026-06-11T01:04:45Z
- Microsoft/DirectXShaderCompiler: c207c7b793957f2d288cb7448516c62808624f71
- Microsoft/DirectXShaderCompiler range: d6e0ca4a0c25..c207c7b79395
- llvm/offload-test-suite: 0d3cc83a8c3570c11026aac1440c50726a671e5a
- llvm/offload-test-suite range: 8c4720026b98..0d3cc83a8c35
FAIL: Feature/CBuffer/vectors-16bit.test
FAIL: Feature/HLSLLib/firstbithigh.64.test
FAIL: Feature/HLSLLib/mad.fp64.test
FAIL: Feature/CBuffer/scalars-16bit.test
FAIL: Feature/CBuffer/arrays-16bit.test
FAIL: Feature/StructuredBuffer/inc_counter_array.test
FAIL: Feature/HLSLLib/mul.int16.test
FAIL: WaveOps/WaveActiveMax.test
FAIL: Feature/HLSLLib/transpose.int16.test
FAIL: Feature/HLSLLib/mul.fp16.test
FAIL: Feature/HLSLLib/transpose.fp16.testSimilarly, for failure-range: $ py ./utils/ci_results.py failure-range windows-amd-clang-warp-d3d12.yaml WaveOps/WaveActiveSum.fp32.test
0 - Run 27349293055 (llvm/llvm-project: 6ff34e926827, test-suite: f788760c7f74): FAIL
1 - Run 27331413794 (llvm/llvm-project: 63e33c6aeed2, test-suite: f788760c7f74): FAIL
3 - Run 27298544113 (llvm/llvm-project: 8d2510ecde24, test-suite: 0d3cc83a8c35): FAIL
7 - Run 27227869184 (llvm/llvm-project: 2b9a8e2d76e1, test-suite: 10ae425589a5): FAIL
15 - Run 27092602665 (llvm/llvm-project: 7df3d9277973, test-suite: bc20b5c4b5ca): PASS
11 - Run 27140683630 (llvm/llvm-project: 5b6236c37658, test-suite: 93b8925e2e89): PASS
9 - Run 27175942016 (llvm/llvm-project: dd87edd1f48b, test-suite: 9bc0142643ef): FAIL
10 - Run 27159468306 (llvm/llvm-project: 3136d62938e8, test-suite: 8c4720026b98): PASS
llvm/llvm-project range: 3136d62938e8..dd87edd1f48b
test suite range: 8c4720026b98..9bc0142643efedit: Updated output 2026-06-11 11:18 AM PDT |
Contributor
|
Looks nice. |
Contributor
|
Consider adding issue #1300 to the description. |
Adds a script that can help inspecting and root cause CI results. We provide two commands, `current-status` and `failure-range`. The first summarizes the results from the most recent run on all CI bots, and the second helps bisect where a particular test started failing. This all works by using the `gh` tool to look at the logs from github workflow runs, with some smarts that have to do with how our runs are set up.
bogner
force-pushed
the
2026-06-10-ci-results
branch
from
4d228a5 to
b71a6ba
Compare
Contributor
Author
Good point. I've updated accordingly |
bob80905
approved these changes
Jun 11, 2026
llvm-beanz
approved these changes
Jun 15, 2026
It seems the code formatter we have in pre-commit CI can't handle f-strings properly...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a script that can help inspecting and root cause CI results. We provide two commands,
current-statusandfailure-range. The first summarizes the results from the most recent run on all CI bots, and the second helps bisect where a particular test started failing.This all works by using the
ghtool to look at the logs from github workflow runs, with some smarts that have to do with how our runs are set up.Fixes #1300