Skip to content

Commit 0f3fa44

Browse files
rjaegersrjaegers
committed
docs: add stp and rtm documents
1 parent 618174e commit 0f3fa44

5 files changed

Lines changed: 305 additions & 1 deletion

File tree

‎.github/workflows/wc-document-generation.yml‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,12 +25,22 @@ jobs:
2525
sudo apt-get update && sudo apt-get install --no-install-recommends -y plantuml
2626
python -m pip install gherkin-official==38.0.0 sbdl==1.21.3
2727
- name: Build & Validate SBDL model
28-
run: sbdl -m compile test/cpp/features/*.feature > amp-devcontainer.sbdl
28+
run: sbdl -m compile test/cpp/integration-tests.bats test/cpp/features/*.feature > amp-devcontainer.sbdl
2929
- name: Generate SRS document
3030
run: sbdl -m template-fill --template docs/templates/software-requirements-specification.md.j2 amp-devcontainer.sbdl > software-requirements-specification.md
31+
- name: Generate STP document
32+
run: sbdl -m template-fill --template docs/templates/software-test-plan.md.j2 amp-devcontainer.sbdl > software-test-plan.md
33+
- name: Generate RTM document
34+
run: sbdl -m template-fill --template docs/templates/requirements-traceability-matrix.md.j2 amp-devcontainer.sbdl > requirements-traceability-matrix.md
3135
- uses: docker://pandoc/extra:3.9.0.0-ubuntu@sha256:72afa9c8d3300e5f10c9c4330e101725687f2179bffd912fb859c6d2ae85de62
3236
with:
3337
args: --template eisvogel --syntax-highlighting idiomatic --number-sections --output software-requirements-specification.pdf software-requirements-specification.md
38+
- uses: docker://pandoc/extra:3.9.0.0-ubuntu@sha256:72afa9c8d3300e5f10c9c4330e101725687f2179bffd912fb859c6d2ae85de62
39+
with:
40+
args: --template eisvogel --syntax-highlighting idiomatic --number-sections --output software-test-plan.pdf software-test-plan.md
41+
- uses: docker://pandoc/extra:3.9.0.0-ubuntu@sha256:72afa9c8d3300e5f10c9c4330e101725687f2179bffd912fb859c6d2ae85de62
42+
with:
43+
args: --template eisvogel --syntax-highlighting idiomatic --number-sections --output requirements-traceability-matrix.pdf requirements-traceability-matrix.md
3444
- uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
3545
with:
3646
name: documents

‎docs/templates/requirements-traceability-matrix.md.j2‎

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
---
2+
title: "Requirements traceability matrix for amp-devcontainer"
3+
author: ["@rjaegers"]
4+
colorlinks: true
5+
date: "March-2026"
6+
keywords: [Traceability, Requirements, RTM, amp-devcontainer]
7+
lang: "en"
8+
titlepage: true
9+
titlepage-color: "0B5ED7"
10+
titlepage-text-color: "FFFFFF"
11+
titlepage-rule-color: "FFFFFF"
12+
titlepage-rule-height: 2
13+
toc: true
14+
toc-own-page: true
15+
...
16+
17+
{% macro reencode(text) -%}
18+
{{ text.encode('utf-8').decode('unicode_escape') }}
19+
{%- endmacro -%}
20+
21+
{%- macro strip_gherkin_prefix(text) -%}
22+
{{ text | replace('Rule: ', '') | replace('Feature: ', '') }}
23+
{%- endmacro -%}
24+
25+
{%- macro display_short(text) -%}
26+
{{ text[:60] }}
27+
{%- endmacro %}
28+
29+
# Introduction
30+
31+
## Purpose
32+
33+
This document provides a requirements traceability matrix (RTM) for amp-devcontainer. It maps each requirement to its associated verification tests, providing evidence of test coverage across the system.
34+
35+
## Abstract
36+
37+
amp-devcontainer is a set of [devcontainers](https://containers.dev/) tailored towards modern, embedded, software development. This traceability matrix traces requirements from the *Software Requirements Specification (SRS)* to their corresponding verification tests as enumerated in the *Software Test Plan (STP)*.
38+
39+
## Document Control
40+
41+
| Property | Value |
42+
|------------------|----------------------|
43+
| Document version | {{ version }} |
44+
| Generation date | {{ generated_at }} |
45+
| Source revision | {{ git_sha }} |
46+
| Source branch/tag| {{ git_ref }} |
47+
| Model | sbdl {{ sbdl_version }} |
48+
49+
# Traceability Matrix
50+
{%- for aspect_id, aspect in sbdl.items() %}
51+
{%- if aspect.type == 'aspect' %}
52+
53+
## {{ reencode(strip_gherkin_prefix(aspect['custom:title'])) }}
54+
55+
| Requirement | Test | Status |
56+
|---|---|---|
57+
58+
{%- if 'requirement' in aspect %}
59+
{%- for req_ref in aspect.requirement %}
60+
{%- set req = sbdl[req_ref.identifier] %}
61+
{%- set req_name = strip_gherkin_prefix(req['custom:title']) | trim %}
62+
63+
{%- set ns_tests = namespace(test_list=[]) -%}
64+
65+
{%- for test_id, test_elem in sbdl.items() -%}
66+
{%- if test_elem.type == 'test' -%}
67+
{%- if 'requirement' in test_elem -%}
68+
{%- for test_req_ref in test_elem.requirement if test_req_ref.identifier == req_ref.identifier -%}
69+
{%- set _ = ns_tests.test_list.append(test_id) -%}
70+
{%- endfor -%}
71+
{%- endif -%}
72+
{%- endif -%}
73+
{%- endfor -%}
74+
75+
{%- if ns_tests.test_list -%}
76+
{%- for test_id in ns_tests.test_list %}
77+
{%- set test_name = strip_gherkin_prefix(sbdl[test_id]['custom:title']) | trim %}
78+
| {{ display_short(req_name) }} | {{ display_short(test_name) }} | Traced |
79+
{%- endfor -%}
80+
{%- else %}
81+
| {{ display_short(req_name) }} | — | **Not covered** |
82+
{%- endif -%}
83+
84+
{%- endfor %}
85+
{%- endif %}
86+
{%- endif %}
87+
{%- endfor %}
88+
89+
# Coverage Summary
90+
91+
{%- set ns_summary = namespace(total_reqs=0, covered_reqs=0, total_tests=0) -%}
92+
93+
{%- for elem_id, elem in sbdl.items() -%}
94+
{%- if elem.type == 'requirement' -%}
95+
{%- set ns_summary.total_reqs = ns_summary.total_reqs + 1 -%}
96+
97+
{%- set ns_covered = namespace(is_covered=false) -%}
98+
{%- for test_id, test_elem in sbdl.items() -%}
99+
{%- if test_elem.type == 'test' and 'requirement' in test_elem -%}
100+
{%- for req_ref in test_elem.requirement if req_ref.identifier == elem_id -%}
101+
{%- set ns_covered.is_covered = true -%}
102+
{%- endfor -%}
103+
{%- endif -%}
104+
{%- endfor -%}
105+
106+
{%- if ns_covered.is_covered -%}
107+
{%- set ns_summary.covered_reqs = ns_summary.covered_reqs + 1 -%}
108+
{%- endif -%}
109+
110+
{%- endif -%}
111+
112+
{%- if elem.type == 'test' -%}
113+
{%- set ns_summary.total_tests = ns_summary.total_tests + 1 -%}
114+
{%- endif -%}
115+
{%- endfor %}
116+
117+
| Metric | Value |
118+
|---|---|
119+
| Total requirements | {{ ns_summary.total_reqs }} |
120+
| Requirements with tests | {{ ns_summary.covered_reqs }} |
121+
| Requirements without tests | {{ ns_summary.total_reqs - ns_summary.covered_reqs }} |
122+
| Total tests | {{ ns_summary.total_tests }} |

‎docs/templates/software-requirements-specification.md.j2‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,50 @@ toc-own-page: true
2020

2121
This document describes the software system requirements for amp-devcontainer.
2222

23+
24+
## Scope
25+
26+
This document covers the requirements for the amp-devcontainer project: a set of devcontainers tailored towards modern, embedded, software development.
27+
28+
The following is in scope:
29+
30+
- Container image flavors for C++ and Rust development
31+
- Tooling for compilation, debugging, static and dynamic analysis
32+
- Compatibility with IDEs, container runtimes, and ci/cd systems
33+
- Security properties of released container images
34+
- Maintainability of the container images and their build system
35+
36+
The following is out of scope:
37+
38+
- Application-level software built using the containers
39+
- Deployment of end-user products
40+
- Requirements for third-party tools and dependencies included in the containers
41+
42+
## References
43+
44+
| Identifier | Title |
45+
|------------|------------------------------------------------------------------------------------------------------------------------------------------|
46+
| RFC 2119 | [Key words for use in RFCs to Indicate Requirement Levels](https://www.rfc-editor.org/rfc/rfc2119) |
47+
| OCI | [Open Container Initiative Image Specification](https://github.com/opencontainers/image-spec/blob/main/spec.md) |
48+
| SLSA | [Supply-chain Levels for Software Artifacts v1.0](https://slsa.dev/spec/v1.0/levels) |
49+
| SemVer | [Semantic Versioning 2.0.0](https://semver.org/spec/v2.0.0.html) |
50+
| DevC | [Development Containers Specification](https://containers.dev/) |
51+
52+
## Document Control
53+
54+
| Property | Value |
55+
|------------------|----------------------|
56+
| Document version | {{ version }} |
57+
| Generation date | {{ generated_at }} |
58+
| Source revision | {{ git_sha }} |
59+
| Source branch/tag| {{ git_ref }} |
60+
| Model | sbdl {{ sbdl_version }} |
61+
62+
This document is generated from a formal requirements model defined in [sbdl](https://sbdl.dev) and versioned alongside the source code in Git.
63+
The authoritative source of change history is the [Git log](https://github.com/philips-software/amp-devcontainer/commits/) for the requirements model files.
64+
65+
Changes to requirements are tracked at the individual requirement level through the version control system and are part of the project's standard change control process.
66+
2367
## Definitions of key words
2468

2569
The key words *MUST*, *MUST NOT*, *REQUIRED*, *SHALL*, *SHALL NOT*, *SHOULD*, *SHOULD NOT*, *RECOMMENDED*, *MAY*, and *OPTIONAL* in this document are to be interpreted as described in [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119).

‎docs/templates/software-test-plan.md.j2‎

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
---
2+
title: "Software test plan for amp-devcontainer"
3+
author: ["@rjaegers"]
4+
colorlinks: true
5+
date: "March-2026"
6+
keywords: [Software, Test, Plan, STP, amp-devcontainer]
7+
lang: "en"
8+
titlepage: true
9+
titlepage-color: "0B5ED7"
10+
titlepage-text-color: "FFFFFF"
11+
titlepage-rule-color: "FFFFFF"
12+
titlepage-rule-height: 2
13+
toc: true
14+
toc-own-page: true
15+
...
16+
17+
{% macro reencode(text) -%}
18+
{{ text.encode('utf-8').decode('unicode_escape') }}
19+
{%- endmacro -%}
20+
21+
{%- macro strip_gherkin_prefix(text) -%}
22+
{{ text | replace('Rule: ', '') | replace('Feature: ', '') }}
23+
{%- endmacro %}
24+
25+
# Introduction
26+
27+
## Purpose
28+
29+
This document describes the test plan for amp-devcontainer.
30+
It defines the test strategy, methodology, and environment, and enumerates the tests that verify each requirement.
31+
32+
For the full requirement descriptions, refer to the *Software Requirements Specification (SRS)*.
33+
For the coverage overview, refer to the *Requirements Traceability Matrix (RTM)*.
34+
35+
## Scope
36+
37+
This document covers the following types of tests:
38+
39+
- **Gherkin verification tests**: Scenario-based tests defined in Gherkin feature files and executed with Playwright. These tests verify behavioral requirements at the system level.
40+
- **BATS integration tests**: Shell-based integration tests defined in BATS (Bash Automated Testing System) files. These tests verify tool availability, version alignment, and end-to-end compilation and analysis workflows.
41+
42+
## References
43+
44+
| Identifier | Title |
45+
|------------|---------------------------------------------------------------------------------------------------------------------------------------|
46+
| SRS | Software Requirements Specification for amp-devcontainer |
47+
| RTM | Requirements Traceability Matrix for amp-devcontainer |
48+
| RFC 2119 | [Key words for use in RFCs to Indicate Requirement Levels](https://www.rfc-editor.org/rfc/rfc2119) |
49+
50+
## Document Control
51+
52+
| Property | Value |
53+
|------------------|----------------------|
54+
| Document version | {{ version }} |
55+
| Generation date | {{ generated_at }} |
56+
| Source revision | {{ git_sha }} |
57+
| Source branch/tag| {{ git_ref }} |
58+
| Model | sbdl {{ sbdl_version }} |
59+
60+
This document is generated from a formal requirements model defined in [sbdl](https://sbdl.dev) and versioned alongside the source code in Git.
61+
62+
## Definitions of key words
63+
64+
The key words *MUST*, *MUST NOT*, *REQUIRED*, *SHALL*, *SHALL NOT*, *SHOULD*, *SHOULD NOT*, *RECOMMENDED*, *MAY*, and *OPTIONAL* in this document are to be interpreted as described in [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119).
65+
66+
# Test environment
67+
68+
Tests are executed in a container built from the amp-devcontainer image. The container provides all necessary compilers, tools, and dependencies.
69+
70+
| Component | Description |
71+
|---|---|
72+
| Container runtime | OCI-compatible container engine (e.g. Docker, Podman) |
73+
| Gherkin test runner | Playwright with Gherkin step definitions |
74+
| Integration test runner | BATS (Bash Automated Testing System) |
75+
| CI platform | GitHub Actions |
76+
77+
# Test methodology
78+
79+
## Gherkin verification tests
80+
81+
Gherkin scenarios follow the Given-When-Then structure. Each scenario exercises a specific capability of the devcontainer by automating interactions with the development environment. Tests are executed via Playwright inside a running devcontainer instance (GitHub Codespace or local). A scenario passes when all of its steps complete without error.
82+
83+
## BATS integration tests
84+
85+
BATS tests verify tool presence, version alignment, and end-to-end workflows (compilation, analysis, formatting) by executing shell commands inside the devcontainer. Each test function runs in isolation. A BATS test passes when the test function exits with code 0.
86+
87+
## Expected results
88+
89+
> **Note**: Structured expected-result data per test is not yet available in the current toolchain. This section will be extended when test result rendering is implemented. Currently, a test is considered passed when it completes without error as described above.
90+
91+
# Tests by requirement area
92+
{%- for aspect_id, aspect in sbdl.items() %}
93+
{%- if aspect.type == 'aspect' %}
94+
95+
## {{ reencode(strip_gherkin_prefix(aspect['custom:title'])) }}
96+
97+
{%- if 'requirement' in aspect %}
98+
{%- for req_ref in aspect.requirement %}
99+
{%- set req = sbdl[req_ref.identifier] %}
100+
101+
### {{ req_ref.identifier }}: {{ reencode(strip_gherkin_prefix(req['custom:title'])) }}
102+
103+
{%- set ns_tests = namespace(has_tests=false) -%}
104+
105+
{%- for test_id, test_elem in sbdl.items() -%}
106+
{%- if test_elem.type == 'test' -%}
107+
{%- if 'requirement' in test_elem -%}
108+
{%- for test_req_ref in test_elem.requirement if test_req_ref.identifier == req_ref.identifier -%}
109+
{%- set ns_tests.has_tests = true %}
110+
- {{ reencode(strip_gherkin_prefix(test_elem['custom:title'])) }}
111+
{%- endfor -%}
112+
{%- endif -%}
113+
{%- endif -%}
114+
{%- endfor -%}
115+
116+
{%- if not ns_tests.has_tests %}
117+
- *No tests traced to this requirement.*
118+
{%- endif -%}
119+
120+
{%- endfor %}
121+
{%- endif %}
122+
{%- endif %}
123+
{%- endfor %}
124+
125+
# Anomaly handling
126+
127+
Test failures are tracked as issues in the project's issue tracker. Each failure is triaged, assigned a severity, and linked to the affected requirement(s). Resolution is verified by re-running the affected test(s).

‎test/cpp/integration-tests.bats‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@ teardown() {
7373
}
7474

7575
@test "valid code input should result in working executable using host compiler" {
76+
# @sbdl test-comp-0001 is test { description is [[[[@-LINE]]]]; requirement is req-comp-0001 }
7677
cmake --preset gcc
7778
cmake --build --preset gcc
7879

0 commit comments

Comments
 (0)