Skip to content

Commit 2d6babb

Browse files
committed
Add scs-0003-v2
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
1 parent 129aefc commit 2d6babb

2 files changed

Lines changed: 90 additions & 2 deletions

File tree

Standards/scs-0003-v1-sovereign-cloud-standards-yaml.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,7 @@
11
---
22
title: Sovereign Cloud Standards YAML
33
type: Procedural
4-
status: Deprecated
5-
deprecated_at: 2026-06-18
4+
status: Draft
65
track: Global
76
description: |
87
SCS-0003 outlines the standards and certification processes for interoperable and sovereign cloud offerings,
Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
---
2+
title: Sovereign Cloud Standards Testing
3+
type: Procedural
4+
status: Draft
5+
track: Global
6+
description: |
7+
SCS-0003 defines concepts central to testing SCS standards and regulates how results may
8+
be obtained and aggregated.
9+
---
10+
11+
## Introduction
12+
13+
Standards
14+
15+
## Concept definitions
16+
17+
A standard can be viewed as a collection of propositions that "must" (or "should") be satisfied
18+
by a test subject (cloud or cluster). The standard is satisfied if all "must" propositions are
19+
satisfied.
20+
21+
A _testcase_ is a collection of propositions that coincide with respect to the following properties:
22+
23+
- whether they can be tested automatically with normal user permissions;
24+
- how often they need to be tested (daily, weekly, monthly, annually);
25+
- whether they are required ("must").
26+
27+
We unambiguously refer to a testcase using a composite identifier consisting of two parts:
28+
29+
- the _scope_: in the context of this standard, just an identifier of a namespace;
30+
for example: `scs-compatible-iaas` (or a UUID)
31+
- the _testcase id_, for example `scs-0100-syntax-check` or `scs-0101-fips-test`.
32+
33+
The scope part is usually clear from the context and therefore omitted.
34+
35+
A _test_ is a testcase or a collection of tests. We refer to a test using the same kind of
36+
composite identifier as for a testcase.
37+
38+
For instance, if we have testcases `scs-0100-syntax-check` and `scs-0100-semantics-check`,
39+
we could define the test `scs-0100-v3` as the collection consisting of these two testcases.
40+
Given further tests `scs-0101-v1`, `scs-0102-v1` etc., we could define the test `scs-0501-v4`
41+
as the collection consisting of `scs-0100-v3`, `scs-0101-v1`, `scs-0102-v1` etc.
42+
43+
A test can be viewed as a collection of propositions; namely, all propositions of all testcases
44+
that are part of the test.
45+
46+
The _result_ of a test is one of the following values:
47+
48+
- `FAIL`: it could be verified that at least one of its propositions is not satisfied;
49+
- `MISS` (missing): for at least one of its propositions, it was not (recently) attempted to verify it;
50+
- `DNF` (did not finish): for at least one of its propositions, it could not be verified whether it is satisfied;
51+
- `PASS`: it could be verified that all its propositions are satisfied.
52+
53+
If multiple items apply, we always opt for the topmost one.
54+
55+
A _score card_ is a data structure that contains the following information:
56+
57+
- Subject: the name of the test subject,
58+
- Scope: the scope of all the testcases referred to in this score card,
59+
- Test results: a mapping that maps testcase ids to results.
60+
61+
A _test report_ is a data structure that contains the information of a score card, plus:
62+
63+
- Creator: who created the report (name of person or version of test suite),
64+
- Check date: when the test was performed,
65+
- Log: free-form text that details the test run.
66+
67+
A _check script_ is a computer program that tests one or more testcases and reports the result per testcase.
68+
69+
The precise form of the data structures (score card, report), as well as the input-output formats of the check scripts is up to the implementation, but it must be well documented.
70+
71+
A _test suite_ is a collection of interoperable check scripts, i.e., check scripts that use the same data structures and I/O formats.
72+
73+
## Regulation
74+
75+
Each standard must be decomposed into testcases. Each testcase should be "atomic" in the following two senses:
76+
77+
- it's clear what specific part of the standard is satisfied or not satisfied;
78+
- the testcase can be reused for multiple versions of the standards.
79+
80+
The latter criterion is a matter of engineering judgment, because it cannot be known in advance how a standard might evolve.
81+
82+
Each testcase id must be prefixed by `scs-XXXX-` where `XXXX` is the document id of the standard; an exception is possible in the rare case when a testcase applies to multiple standards.
83+
84+
A check script that can test multiple testcases should provide the option to select which testcases to run.
85+
86+
A list of reports from a certain time frame can be merged into an aggregate score card, provided that the following conditions are satisfied:
87+
88+
- for each testcase, the result must be taken from the most recent report containing that testcase,
89+
- if an additional report from the same time frame is known to exist, all its testcases must be contained in more recent reports from the list.

0 commit comments

Comments
 (0)