Skip to content

Commit db1b863

Browse files
garloffgarloff
authored
Merge branch 'main' into feat/more-gpus
2 parents 026042b + 7a2d25d commit db1b863

24 files changed

Lines changed: 372 additions & 731 deletions

‎.zuul.d/config.yaml‎

Lines changed: 31 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,9 @@
66
periodic-daily:
77
jobs:
88
- scs-check-iaas
9-
- scs-check-kaas
9+
- scs-check-kaas1
10+
- scs-check-kaas2
11+
- scs-check-kaas3
1012
periodic-hourly:
1113
jobs:
1214
- scs-check-scs2-main
@@ -16,14 +18,20 @@
1618
check:
1719
jobs:
1820
- scs-check-adr-syntax
21+
- semaphore:
22+
name: semaphore-iaas
23+
max: 1 # run at most one iaas job at the same time
24+
- semaphore:
25+
name: semaphore-kaas
26+
max: 1 # run at most one kaas job at the same time
1927
- job:
2028
name: scs-check-adr-syntax
2129
parent: base
2230
nodeset: pod-fedora-40
2331
pre-run: playbooks/pre.yaml
2432
run: playbooks/adr_syntax.yaml
2533
- job:
26-
name: scs-check-scs2
34+
name: scs-check-base
2735
parent: base
2836
secrets:
2937
- name: clouds_conf
@@ -44,6 +52,11 @@
4452
run: playbooks/compliance_check.yaml
4553
post-run:
4654
- playbooks/post_cloud.yaml
55+
- job:
56+
name: scs-check-scs2
57+
parent: scs-check-base
58+
semaphores:
59+
- semaphore-iaas
4760
- job:
4861
name: scs-check-scs2-main
4962
parent: scs-check-scs2
@@ -59,17 +72,30 @@
5972
vars:
6073
preset: iaas
6174
- job:
62-
name: scs-check-kaas
63-
parent: scs-check-scs2-main
75+
name: scs-check-kaas1
76+
parent: scs-check-base
77+
branches: main
78+
semaphores:
79+
- semaphore-kaas
6480
# timeout:
6581
# a) these tests take a lot of time, I'm afraid, particularly Sonobuoy
6682
# b) keep in mind that this job covers ALL test subjects (at most 4 in parallel)
6783
timeout: 21600 # 6 hrs -- 5 hrs was almost sufficient (reports came through sometimes)
6884
attempts: 1 # this job is heavy, and retries aren't very promising
6985
vars:
70-
preset: kaas
86+
preset: kaas1
7187
iaas: false
7288
kaas: true
7389
section: '.auto' # only do 'heavy' tests on Saturdays
7490
do_provision: true
7591
do_cleanup: false
92+
- job:
93+
name: scs-check-kaas2
94+
parent: scs-check-kaas1
95+
vars:
96+
preset: kaas2
97+
- job:
98+
name: scs-check-kaas3
99+
parent: scs-check-kaas1
100+
vars:
101+
preset: kaas3

‎.zuul.d/secure.yaml‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -458,16 +458,16 @@
458458
/Lt6+yMJMB7AUVpwXkI7cMmf+QXbDnnE6eX0UBosTM6MjdA/zhkjJ6PK9E0FnngW4HRDZ
459459
eAiVSQiN4CSa9bylrwLhvarAEhR+5avRv4yTurgAYe6I6Gh05pBPCgTXmCNxPU=
460460
scaleup_api_server: !encrypted/pkcs1-oaep
461-
- H4F/zi6Lp/CbkHPG4tEzhFVRprH6uxVqoOirUZNKOg1pDI33ilXG4K99O/CH9dCqbFiV0
462-
GnI4p8b0Csho6S63DEuMikjXBLfNWsd2BncQ4gu4bGhHef3bkCjEmybk3AmEyA789vRNS
463-
o6xjZUAJB0ZEQeuWudSo28fJDGxGOVcVW0Sda+OSk0K3qftucg047KZdJ5bWZmHTPvOha
464-
4R2+nsbnecAQJtMDWVERPYTD62pF4CAHD6XVf9mUuZNnLYibl5uQz0t6JZQANu83pFYDZ
465-
C5Z7T4/UJxKlyBpL2XtF/cHuKavEP41cheD/ihG1EI9Yf3InFCRllbIGzMj8fq8d4bZKV
466-
tC1iui2RrL3s38GrOyyGE7odyHlvl08zbrJTkWb5UcI20jPxBCxHAvlqIWjl+9Ite7OuM
467-
nXq62IgXUqVWWgrGWRINyJlvRgcQPxCVq7ty3OMB1Vp4R0ldDRWWR9+PozmK86fRXYatE
468-
cgRi579JMBvKjbP9hKcZOkb12PVyZTTOLmyepfMAqZVpBULtpNza26ysCl5Wu14+0o3bh
469-
d6Kst5qeZz4P8QjJe2UQU67B8JcX88zbpNYM1uxLhtXlAk0/8Hd8KWm+KiU/NfNzgr7jd
470-
jxRhMEdHUuMOKHaSR/hwJs7kWp1RVR6l7L+SXv02VrYgzysGCiGqXhV9FwrEQw=
461+
- VCeER+5TfxyzOnvIWrAx30FRdwa4I837hPnELGWWmmzdyPQCKVqP4dKEX09nmzwabLnWF
462+
WEzMhG9tHsIPBImP2x6VMAqwGsocAdS+Xw73gJXJm0P79gc2xqbnL8BpirraAH+TpCG+G
463+
eYcQKzfKfhIViVix7WJfeqodN6sB3oaAodlcvjMpqeYL+USQsZj+ejZZc61P0ec8AybXd
464+
lCxuRrdutks6aJlnIaQ9o1G+40DUzljd6RrrGQDPfQOx8XZCaN+eskpox/iKVX6Fso23U
465+
Zv8ZCQg2FsT9M9s2GC/KcLgQcHVIkKTYpr6Fnx4ymoxfCE4D2l2B6hK6PcT8I4OAMDZKn
466+
MtV8wSsSbnohUpCGN8qMyjTf5q2WHr55BWaYmvRVkAwm+eAEd2/EBr6FSXss9C0hKNznr
467+
w60LZhXC5QCXvc3tS+A+8s913luzQI8zmlQCSHHqIJujtLNhP/BO/7EpR9GMHD9KZxP17
468+
IqElVTiWYJX2tzx309+G4zhw5L8qwG9FvaDVr9DBdv1mgrKl7QTi5SbfQMn4DExCCaJmR
469+
kglEjdtYoHoulJP0KD9a1Sfrg0sQDPBag44cjORfN8mvzPcI2QLAz+iClZhi2pHVnwEZH
470+
k49ZE1+gp/PUdlFPDdhHVAJgB4tzZEXPOrbgUGCIqqNam9IwzOrRuFR8fIvXVo=
471471
scaleup_ca_data: !encrypted/pkcs1-oaep
472472
- CDar1TQ3Q4HrBXIDwVFvzL0FgPFy7r3K4joQmKnIbCJZ9pmbtrGGSD7xOsJnfIm5e89ZK
473473
g4D654VfvK4+upredcVP5MRkfXY/SKfh9glR9okQMpsRS0T5l3kS+RpZOLPCuvvit3i1h

‎Standards/scs-0123-v2-services.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ title: Mandatory and Supported IaaS Services
33
type: Standard
44
status: Draft
55
track: IaaS
6+
replaces: scs-0123-v1-mandatory-and-supported-IaaS-services.md
67
---
78

89
## Introduction

‎Standards/scs-0210-v2-k8s-version-policy.md‎

Lines changed: 2 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -66,10 +66,10 @@ the provided Kubernetes versions should be kept up-to-date with new upstream rel
6666
- New versions MUST be tested before being rolled out on productive infrastructure;
6767
at least the [CNCF E2E tests][cncf-conformance] should be passed beforehand.
6868

69-
At the same time, providers must support Kubernetes versions at least as long as the
69+
At the same time, providers must support and offer Kubernetes versions at least as long as the
7070
official sources as described in [Kubernetes Support Period][k8s-support-period]:
7171

72-
- Kubernetes versions MUST be supported as long as the official sources support them
72+
- Kubernetes versions MUST be supported and offered as long as the official sources support them
7373
according to the [Kubernetes Support Period][k8s-support-period] and their end-of-life
7474
date according to the [Kubernetes Releases page][k8s-releases].
7575
- It is RECOMMENDED to not support versions after this period in order to not encourage
@@ -80,17 +80,6 @@ official sources as described in [Kubernetes Support Period][k8s-support-period]
8080
All documents regarding versioning, releases, etc. for the official Kubernetes projects can
8181
be found on the [Kubernetes Releases page][k8s-releases].
8282

83-
## Conformance Tests
84-
85-
The script `k8s_version_policy.py` requires a kubeconfig file with connection details for
86-
a set of existing Kubernetes clusters that should be checked, with each of these clusters
87-
representing one of the currently supported upstream Kubernetes releases.
88-
It will check the encountered cluster versions according to the rules of this standard.
89-
Rule violations will be reported on various logging channels: ERROR for mandatory rules
90-
and INFO for recommended rules.
91-
The script will exit with a non-zero status if a mandatory rule has been violated or if
92-
the test could not be performed.
93-
9483
[k8s-releases]: https://kubernetes.io/releases/
9584
[k8s-release-cycle]: https://kubernetes.io/releases/release/#the-release-cycle
9685
[k8s-release-cadence]: https://kubernetes.io/releases/patch-releases/#cadence

‎Tests/config.toml‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,11 +40,51 @@ scopes = [
4040
"scs-compatible-kaas",
4141
]
4242
subjects = [
43+
"noris-1.33",
4344
"noris-1.34",
45+
"noris-1.35",
4446
"scaleup-1.33",
4547
"scaleup-1.34",
48+
"scaleup-1.35",
4649
"syself-1.33",
4750
"syself-1.34",
51+
"syself-1.35",
52+
]
53+
workers = 16
54+
55+
56+
[presets.kaas1]
57+
scopes = [
58+
"scs-compatible-kaas",
59+
]
60+
subjects = [
61+
"noris-1.33",
62+
"scaleup-1.33",
63+
"syself-1.33",
64+
]
65+
workers = 4
66+
67+
68+
[presets.kaas2]
69+
scopes = [
70+
"scs-compatible-kaas",
71+
]
72+
subjects = [
73+
"noris-1.34",
74+
"scaleup-1.34",
75+
"syself-1.34",
76+
]
77+
workers = 4
78+
79+
80+
[presets.kaas3]
81+
scopes = [
82+
"scs-compatible-kaas",
83+
]
84+
subjects = [
85+
"noris-1.35",
86+
"scaleup-1.35",
87+
"syself-1.35",
4888
]
4989
workers = 4
5090

‎Tests/iaas/openstack_test.py‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -231,7 +231,7 @@ def harness(name, *check_fns):
231231
print(f"{name}: {result}")
232232

233233

234-
def run_sanity_checks(container):
234+
def run_preflight_checks(container):
235235
# make sure that we can connect to the cloud and that the user doesn't have elevated privileges
236236
# the former would lead to each testcase aborting with a marginally useful message;
237237
# the latter would lead to scs_0116_permissions aborting, which we don't want to single out
@@ -278,7 +278,13 @@ def main(argv):
278278
sys.exit(1)
279279

280280
c = make_container(cloud)
281-
run_sanity_checks(c)
281+
try:
282+
run_preflight_checks(c)
283+
except Exception:
284+
logger.critical("Pre-flight checks failed. Reporting all testcases as ABORT.")
285+
for testcase in testcases:
286+
print(f"{testcase}: ABORT")
287+
raise
282288
for testcase in testcases:
283289
harness(testcase, lambda: getattr(c, testcase.replace('-', '_')))
284290
return 0

‎Tests/iaas/scs_0101_entropy/entropy_check.py‎

Lines changed: 22 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -127,28 +127,28 @@ def compute_scs_0101_rngd(collected_vm_output, image_name):
127127
def compute_scs_0101_fips_test(collected_vm_output, image_name):
128128
"""This test ensures that the 'fips test' via `rngtest` is passed on a test VM."""
129129
lines = collected_vm_output['fips-test']
130-
try:
131-
fips_data = '\n'.join(lines)
132-
failure_re = re.search(r'failures:\s\d+', fips_data, flags=re.MULTILINE)
133-
if failure_re:
134-
fips_failures = failure_re.string[failure_re.regs[0][0]:failure_re.regs[0][1]].split(" ")[1]
135-
if int(fips_failures) <= 3:
136-
return True # strict test passed
137-
logger.info(
138-
f"VM '{image_name}' didn't pass the strict FIPS 140-2 testing. "
139-
f"Expected a maximum of 3 failures, got {fips_failures}."
140-
)
141-
if int(fips_failures) <= 5:
142-
return True # lenient test passed
143-
logger.error(
144-
f"VM '{image_name}' didn't pass the FIPS 140-2 testing. "
145-
f"Expected a maximum of 5 failures, got {fips_failures}."
146-
)
147-
else:
148-
logger.error(f"VM '{image_name}': failed to determine fips failures")
149-
logger.debug(f"stderr following:\n{fips_data}")
150-
except BaseException:
151-
logger.critical(f"Couldn't check VM '{image_name}' requirements", exc_info=True)
130+
fips_data = '\n'.join(lines)
131+
failure_re = re.search(r'failures:\s\d+', fips_data, flags=re.MULTILINE)
132+
if not failure_re:
133+
# It seems possible that 'failures: 0' is just omitted, and we could check for
134+
# 'successes: 1000' to verify that, but I have observed this only once in many
135+
# many runs over multiple years. I'm inclined to label this 'inconclusive'
136+
# instead of making the code more complex and risking false certainty.
137+
logger.debug(f"failed to determine fips failures; stderr following:\n{fips_data}")
138+
raise RuntimeError(f"VM '{image_name}': failed to determine fips failures")
139+
fips_failures = failure_re.string[failure_re.regs[0][0]:failure_re.regs[0][1]].split(" ")[1]
140+
if int(fips_failures) <= 3:
141+
return True # strict test passed
142+
logger.info(
143+
f"VM '{image_name}' didn't pass the strict FIPS 140-2 testing. "
144+
f"Expected a maximum of 3 failures, got {fips_failures}."
145+
)
146+
if int(fips_failures) <= 5:
147+
return True # lenient test passed
148+
logger.error(
149+
f"VM '{image_name}' didn't pass the FIPS 140-2 testing. "
150+
f"Expected a maximum of 5 failures, got {fips_failures}."
151+
)
152152
return False # any unsuccessful path should end up here
153153

154154

‎Tests/kaas/k8s-version-recency/config.yaml.template‎

Lines changed: 0 additions & 24 deletions
This file was deleted.

0 commit comments

Comments
 (0)