Skip to content

record aborted testcases, part 1 - #975

Merged
mbuechse merged 31 commits into
mainfrom
issue/889
Sep 8, 2025
Merged

record aborted testcases, part 1#975
mbuechse merged 31 commits into
mainfrom
issue/889

Conversation

@mbuechse

@mbuechse mbuechse commented Aug 15, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

closing #889

also addresses some of #942 and #951

@mbuechse

mbuechse commented Aug 28, 2025

Copy link
Copy Markdown
Contributor Author

@depressiveRobot the markdown-link-check fails, but this is because the link in question refers to the main branch; ironically, merging this branch will immediately repair this problem. Also, be advised that merging is entirely possible (the check is not a blocker).

@mbuechse
mbuechse marked this pull request as ready for review August 29, 2025 13:53
depressiveRobot
depressiveRobot reviewed Sep 5, 2025
View reviewed changes

@depressiveRobot depressiveRobot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing that. I welcome the approach of catching all exceptions outside the actual test method.

However, the origin for this change still seems to be not fixed: if a test didn't finish cause of an unhandled exception (now ABORT), it is still not linked in the compliance monitor:

Screenshot 2025-09-05 at 18 06 04

Here is the report with one aborted check which I used to test this:
scs2-report.yaml

@mbuechse

mbuechse commented Sep 5, 2025

Copy link
Copy Markdown
Contributor Author

@depressiveRobot I don't see the URL here. You know you have to look at /detail_full, don't you?

@depressiveRobot

depressiveRobot commented Sep 8, 2025

Copy link
Copy Markdown
Contributor

Thanks for pointing that out. I haven't fully internalized that yet.

When using /page/table_full I can see the logs and the catched exception as expected.

From my point of view this can be merged now.

@mbuechse
record aborted testcases, part 1
26b63f5 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
fix typo
6ac57e6 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
removed help on obsolete command-line switches
1073fbd 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
factor out test logic into separate module
a3f562a 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
rename subdirectory to make imports possible, also more telling name
127118a 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
fixup: adapt scs-compatible-iaas.yaml after rename
01d4dbe 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
fixup: adapt adr_syntax.yaml after rename
fbf1bc5 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
add openstack_test.py
17b92b4 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
fixup: acquiesce flake8
7de15b1 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
adapt entropy-check parts
38d78ac 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
fixup: satisfy flake8 (actually good points)
ba10feb 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
revise outdated usage text
9ed2215 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
change channel back to error when testcase is not passed
9287559 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
adapt check pertaining to scs-0102-image-metadata
aa17fbb 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
adapt check pertaining to scs-0103-standard-flavors
d6619d0 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
Adapt check pertaining to scs-0104-standard-images
0cb500b 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
adapt checks pertaining to scs-0114-volume-types
81e6b6c 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
Adapt check pertaining to scs-0115-security-groups
e181041 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
Pacify flake8
f1a9170 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
Adapt check pertaining to scs-0116-key-manager
64dc965 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
Adapt check pertaining to scs-0117-volume-backup
fbcb7a9 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
fixup: forgot to update scs-compatible-iaas.yaml
7185584 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
mbuechse and others added 9 commits September 8, 2025 12:57
@mbuechse
Adapt check pertaining to scs-0123-mandatory-services
637fb56 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
Remove redundant stand-alone check scripts, improve documentation
6ddcfec 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
started adapting testing notes
6b30de7 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
adapt testing notes, contd.
6e6ea9d 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
adapt testing notes, contd.
c14e7a7 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
added missing doc
95515ab 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
adapt testcase evaluation and reporting
49964f3 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
remove obsolete documentation and scripts
30f1f92 
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse @depressiveRobot
Fix typos; thanks @depressiveRobot
dae1f8c 
Co-authored-by: Marvin Frommhold <depressiveRobot@users.noreply.github.com>
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse
mbuechse merged commit 9dadf5a into main Sep 8, 2025
9 of 10 checks passed
@mbuechse
mbuechse deleted the issue/889 branch September 8, 2025 12:04
@depressiveRobot depressiveRobot mentioned this pull request Sep 18, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@depressiveRobot depressiveRobot depressiveRobot approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@mbuechse @depressiveRobot