Merge remote-tracking branch 'origin/main' into template_inspection_needs

# Conflicts:
#	src/extensions/score_metamodel/metamodel.yaml

Author: FScholPer

.github/workflows/renovate.yml

@@ -69,7 +69,7 @@ http_file(
     urls = ["https://github.com/withered-magic/starpls/releases/download/v0.1.21/starpls-linux-amd64"],
 )
 
-bazel_dep(name = "score_process", version = "1.6.0")
+bazel_dep(name = "score_process", version = "2.0.1")
 
 # Provide the tools from the devcontainer to Bazel
 bazel_dep(name = "score_devcontainer", version = "1.7.0")

MODULE.bazel

@@ -4,111 +4,141 @@
 ## Test Function
 The functionality of the Sphinx build rules can be verified with test rst files.
 
-The function *test_check_rules* in *test_rules_file_based.py* is executed for
+The function *test_rst_files* in *test_rules_file_based.py* is executed for
 each rst file in the directory *rst*.
 It creates a SphinxTestApp and a document source folder with an index.rst file
 that contains a toctree with the given rst file.
 
 It uses the SphinxTestApp to build the documentation and checks for the
 **expected/not expected** warnings.
 
+**It has it's OWN conf.py, so if changes need to be made ensure they are made in there**
+
 ## Create a test rst file
-To add a new test case create a new rst file in the rst directory.
+To add a new test case either create a new rst file in the rst directory,
+or add it to an existing one if it tests something similar
 The test files can also be organized in a subfolder structure below directory rst.
-The test files are expected to contain the following format:
 
-    #CHECK: <check functions>
+Each test file consists of two parts:
 
-    #EXPECT[+x]: <warning message>
-    #EXPECT-NOT[+x]: <warning message>
+1. Exactly **one** `test_metadata` need at the top of the file. It declares which
+   check(s) to run and links the test to the requirements it verifies.
+2. One or more Sphinx-Needs directives. A need that should (or should not) emit a
+   warning carries an `:expect:` / `:expect_not:` option describing that warning.
 
-    <need information>
+### The `test_metadata` need
 
-**\<check functions>**<br>
-Check functions (comma separated) to be used for the Sphinx build. Following
-                    warnings will only be generated by these check functions.
-                    Only one CHECK statement per file. Usually at the very top.
-                    If CHECK is not provided, all checks are enabled.
+`test_metadata` is now a real Sphinx-Needs directive (not a comment). There must
+be exactly one per file:
 
-**\<warning message>**<br>
-Message text which is expected/not expected during the
-                    Sphinx build to be shown.
-                    This message is checked for the Sphinx-Needs directive
-                    specified after the EXPECT/EXPECT-NOT statement.
+.. code-block:: rst
 
-This message needs a '[+x]'offset after the 'EXPECT/-NOT' that should point to the need
-that should (not) emit the warning.
+    .. test_metadata::
+       :id: test_metadata__<unique_name>
+       :partially_verifies_list: <requirement id(s)>
+       :test_type: requirements_based
+       :derivation_technique: requirements_based
 
-**\<need information>**<br>
-One or more Sphinx-Needs directives needed for the
-                    Sphinx document build
+       <description of what this file tests>
 
-**Example:**
+**\<check functions>**
+All checks are run in each rst test file to ensure no contradiction and make testing easier / more complete
 
-    #CHECK: check_options
-    #EXPECT[+2]: std_wp__test__abcd: is missing required attribute: `status`.
+**\<requirement id(s)>**`:partially_verifies_list:` / `:fully_verifies_list:`<br>
+The requirement(s) this test verifies. At least one of `:partially_verifies_list:`
+or `:fully_verifies_list:` must be provided, otherwise the test fails.
 
-    .. std_wp:: Test requirement
-        :id: std_wp__test__abcd
+**`:test_type:` / `:derivation_technique:`**<br>
+Describe how the test was derived. They are attached as properties to the test
+result.
 
-This example verifies that the warning message
-*std_wp__test__abcd: is missing required attribute: \`status\`*
-is shown during the Sphinx build. Only the *check_options* check is enabled.
+### Needs with expectations
 
-With the '[+2]' after the 'EXPECT' we tell the parser that we want this warning
-to be emitted and checked for 2 lines underneath
+The `:expect:` / `:expect_not:` options live **on the need that is being tested**.
+There is no separate statement and no `[+x]` line offset anymore &mdash; the warning is
+matched against the need's own location.
 
-There is multiple things that are not allowed for example, you need to have
-a new line between the EXPECT/-NOT and the need that it refers to
+**\<warning message>** &mdash; `:expect:` / `:expect_not:`<br>
+Message text which is expected / not expected during the Sphinx build for this
+need. You only need to match a unique substring, not the full message.
 
-**Negative Example**
+A need **without** `:expect:` or `:expect_not:` is treated as a setup /
+prerequisite need (for example a link target) and is not asserted against.
 
+**Example:**
+
+.. code-block:: rst
+
+    .. test_metadata::
+       :id: test_metadata__mandatory_options
+       :partially_verifies_list: tool_req__docs__status
+       :test_type: requirements_based
+       :derivation_technique: requirements_based
+
+       Tests that mandatory options are enforced.
 
-    #EXPECT-NOT[+1]: std_wp__test__abcd: is missing required attribute: `status`.
     .. std_wp:: Test requirement
-        :id: std_wp__test__abcd
+       :id: std_wp__test__abcd
+       :expect: std_wp__test__abcd: is missing required attribute: `status`.
 
-This will error and let you know that an offset of '1' is not allowed and you
-need to add a new line beneath the Warning Statement
+This example verifies that the warning message
+*std_wp__test__abcd: is missing required attribute: \`status\`* is shown during
+the Sphinx build.
 
-## Graph check tests
+**Expect-not example:**
 
-Graph checks (defined in `metamodel.yaml` under `graph_checks:`) are all executed
-by a single Python function `check_metamodel_graph`. Use that function name in `#CHECK:`:
+    .. std_wp:: Test requirement
+       :id: std_wp__test_options__abce
+       :status: active
+       :expect_not: attribute
 
-    #CHECK: check_metamodel_graph
+Here the need is fully valid, so we assert that no warning containing the
+substring *attribute* is emitted for it.
 
-Graph check test files live in `tests/rst/graph/`.
+> **Note:** RST comments used to separate or describe cases (for example
+> `.. All required options are present`) must not contain `::`, otherwise the
+> parser counts them as needs.
 
 ### Warning message format
 
-`check_metamodel_graph` emits warnings in this shape:
+You only need to match a unique substring, not the full message. For example:
 
-    {need_id}: Parent need `{parent_id}` does not fulfill condition `{condition}`. Explanation: {explanation}
+    :expect: wp__bad: Parent need `std_req__aspice_40__bp`
 
-You only need to match a unique substring, not the full message. For example:
+.. hint::
 
-    #EXPECT[+2]: wp__bad: Parent need `std_req__aspice_40__bp`
+   For `:expect:` use as much as possible of the error message, however for `:expect_not:` use as little as possible
+   to catch all other issues that MIGHT occur and to avoid for typos etc.
 
 ### Setup needs
 
-Prerequisite needs (link targets, shared fixtures) belong at the top of the file
-with no `EXPECT`/`EXPECT-NOT` annotation. The framework only checks annotated lines,
-so unannotated needs are invisible to the assertion logic.
+Prerequisite needs (link targets, shared fixtures) carry no `:expect:` /
+`:expect_not:` option. The framework only asserts on needs that have one of these
+options, so unannotated needs are invisible to the assertion logic.
 
 ### The "exempt" case
 
 A need that does not meet the `condition:` in the YAML check definition is not
-selected by the check at all — no warning is emitted. Test this with `EXPECT-NOT`:
+selected by the check at all &mdash; no warning is emitted. Test this with
+`:expect_not:`:
 
-    #EXPECT-NOT[+2]: <something from the explanation>
+.. code-block:: rst
 
     .. workproduct:: No link at all
        :id: wp__no_link
+       :expect_not: <something from the explanation>
 
 ### Full example (graph check)
 
-    #CHECK: check_metamodel_graph
+.. code-block:: rst
+
+    .. test_metadata::
+       :id: test_metadata__graph_aspice_40
+       :partially_verifies_list: gd_req__aspice_40__parent_link
+       :test_type: requirements_based
+       :derivation_technique: requirements_based
+
+       Verifies the ASPICE 40 IIC parent-link graph check.
 
     .. std_req:: ASPICE 40 IIC requirement
        :id: std_req__aspice_40__iic_1
@@ -120,23 +150,20 @@ selected by the check at all — no warning is emitted. Test this with `EXPECT-N
 
     .. Positive: links to allowed target — no warning.
 
-    #EXPECT-NOT[+2]: ASPICE 40 IIC
-
     .. workproduct:: Valid workproduct
        :id: wp__valid
        :complies: std_req__aspice_40__iic_1
+       :expect_not: ASPICE 40 IIC
 
     .. Exempt: no complies link — condition not met, check skipped.
 
-    #EXPECT-NOT[+2]: ASPICE 40 IIC
-
     .. workproduct:: Workproduct without link
        :id: wp__no_link
+       :expect_not: ASPICE 40 IIC
 
     .. Negative: links to disallowed target — warning expected.
 
-    #EXPECT[+2]: wp__bad: Parent need `std_req__aspice_40__bp_1`
-
     .. workproduct:: Invalid workproduct
        :id: wp__bad
        :complies: std_req__aspice_40__bp_1
+       :expect: wp__bad: Parent need `std_req__aspice_40__bp_1`