Skip to content

Commit defaf01

Browse files
clanghansclanghans
committed
docs: add Sphinx documentation site
Add full documentation under docs/ covering: - How-to guides: get started, write tests, plugin usage - Reference: Bazel macros/rules, plugin CLI args, Target API - Concepts: architecture overview with PlantUML diagram, plugin system, capability model, plugin loading order - Stub sections for manual, release, safety_mgt, security_mgt, verification_report per S-CORE module folder structure spec
1 parent a05e144 commit defaf01

16 files changed

Lines changed: 1352 additions & 0 deletions

File tree

‎docs/concepts/architecture.rst‎

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
..
2+
# *******************************************************************************
3+
# Copyright (c) 2026 Contributors to the Eclipse Foundation
4+
#
5+
# See the NOTICE file(s) distributed with this work for additional
6+
# information regarding copyright ownership.
7+
#
8+
# This program and the accompanying materials are made available under the
9+
# terms of the Apache License Version 2.0 which is available at
10+
# https://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# SPDX-License-Identifier: Apache-2.0
13+
# *******************************************************************************
14+
15+
.. _itf_architecture:
16+
17+
Architecture
18+
============
19+
20+
This page explains the core design decisions in ITF: the target abstraction
21+
layer, the capability system, the plugin lifecycle, and how ITF integrates
22+
bidirectionally with Bazel.
23+
24+
.. plantuml:: itf_architecture.puml
25+
26+
Target abstraction layer
27+
------------------------
28+
29+
The central concept in ITF is the ``Target``. A target represents the device
30+
or environment under test — a Docker container, a QEMU virtual machine, or
31+
real hardware. All target types expose the same interface, so test code does
32+
not need to know which environment it runs on.
33+
34+
.. code-block:: python
35+
36+
class Target:
37+
def execute(self, command): ...
38+
def upload(self, local_path, remote_path): ...
39+
def download(self, remote_path, local_path): ...
40+
def restart(self): ...
41+
def get_capabilities(self) -> Set[str]: ...
42+
43+
A test that calls ``target.execute("uname -a")`` runs unchanged against a
44+
Docker container or a QEMU VM. The target type is determined at build time by
45+
the ``plugins`` attribute on ``py_itf_test``, and at run time by the CLI
46+
args (e.g. ``--docker-image``) that configure the chosen plugin.
47+
48+
Capability system
49+
-----------------
50+
51+
Different target environments support different operations. A plain Docker
52+
container supports ``exec`` and file transfer but not SSH or SFTP unless an
53+
SSH server is installed. A QEMU VM provides SSH, SFTP, and network-level
54+
operations.
55+
56+
Each ``Target`` subclass declares its capabilities, either by passing them
57+
to the base constructor or by relying on ``Target.REQUIRED_CAPABILITIES``
58+
(``exec``, ``file_transfer``, ``restart``), which is always merged in.
59+
``DockerTarget`` uses only the required capabilities, so it passes no
60+
extras:
61+
62+
.. code-block:: python
63+
64+
class DockerTarget(Target):
65+
def __init__(self, container):
66+
super().__init__() # capabilities come from REQUIRED_CAPABILITIES
67+
self.container = container
68+
69+
Tests can be guarded against targets that lack required capabilities using the
70+
``@requires_capabilities`` decorator:
71+
72+
.. code-block:: python
73+
74+
from score.itf.plugins.core import requires_capabilities
75+
76+
@requires_capabilities("ssh", "sftp")
77+
def test_file_roundtrip(target):
78+
...
79+
80+
If the active target does not provide all listed capabilities, pytest skips the
81+
test with a clear message. This keeps test suites portable: the same file can
82+
run against Docker for fast feedback and against a QEMU VM for full-system
83+
integration, skipping tests that do not apply.
84+
85+
Tests can also query capabilities at runtime and branch accordingly:
86+
87+
.. code-block:: python
88+
89+
def test_adaptive(target):
90+
if target.has_capability("ssh"):
91+
with target.ssh() as ssh:
92+
ssh.execute_command("echo hello")
93+
else:
94+
exit_code, _ = target.execute("echo hello")
95+
96+
Plugin lifecycle
97+
----------------
98+
99+
Each plugin contributes a ``target_init`` pytest fixture. ITF's core plugin
100+
calls this fixture to obtain the target instance, then wraps it in the
101+
``target`` fixture that test functions receive.
102+
103+
The lifecycle for a single test is:
104+
105+
1. **Setup**: The plugin's ``target_init`` fixture starts the target
106+
(spins up a container, boots a QEMU VM, connects to hardware).
107+
2. **Test execution**: The test function receives the ``target`` fixture
108+
and exercises the system under test.
109+
3. **Teardown**: ``target_init`` tears down the target (stops the
110+
container, shuts down the VM).
111+
112+
With ``--keep-target``, steps 1 and 3 run once per session instead of once
113+
per test function. This is faster but means tests share target state, so it
114+
should only be used when tests are designed to be order-independent.
115+
116+
**Plugin loading order is deterministic but should not be relied upon.**
117+
The core plugin is always registered first. The remaining plugins are
118+
registered with pytest in the exact order they are listed in
119+
``py_itf_test.plugins``. While this order is stable, plugins are designed
120+
to be independent of each other — no plugin should depend on another
121+
plugin's initialisation having completed first.
122+
123+
Why a plugin-based design
124+
--------------------------
125+
126+
Plugin-based design was chosen for three reasons:
127+
128+
**Separation of concerns.** Target management logic (starting containers,
129+
booting VMs) is entirely isolated from test logic. A test that calls
130+
``target.execute()`` has no dependency on Docker or QEMU APIs.
131+
132+
**Extensibility without forking.** Custom targets (real hardware, emulators,
133+
cloud VMs) are added by implementing ``Target`` and ``target_init`` in a new
134+
plugin. No changes to the ITF core are needed.
135+
136+
**Bazel-native composition.** Because plugins are declared as Bazel targets
137+
with ``py_itf_plugin``, they carry their own Python libraries, data files,
138+
and CLI args. Combining plugins — for example Docker + DLT — is as simple as
139+
listing both labels in ``py_itf_test.plugins``. Bazel resolves transitive
140+
dependencies automatically.
141+
142+
Bidirectional Bazel integration
143+
---------------------------------
144+
145+
ITF integrates with Bazel in both directions:
146+
147+
**Build-time** (Bazel → ITF): The ``py_itf_test`` symbolic macro creates a
148+
``py_test`` binary that bundles the test code and all plugin Python
149+
libraries. Plugin CLI args (e.g. ``--docker-image``, paths from
150+
``$(location ...)``) are resolved at analysis time and baked into a launcher
151+
script. This means test hermetically carry their full dependency graph,
152+
including container images or QEMU images referenced via Bazel labels.
153+
154+
**Run-time** (ITF → Bazel): ITF uses Bazel's runfiles mechanism to locate
155+
data files at runtime. The ``$(location ...)`` substitution in ``args``
156+
produces runfiles-relative paths that work regardless of where Bazel places
157+
files in the output tree. Test results are reported via JUnit XML to
158+
``$XML_OUTPUT_FILE``, integrating with Bazel's native test reporting and
159+
caching.
160+
161+
This design means ITF tests participate fully in Bazel's incremental build
162+
and caching: a test is only re-run if its source, its dependencies, or its
163+
configuration changes.

‎docs/concepts/index.rst‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
..
2+
# *******************************************************************************
3+
# Copyright (c) 2026 Contributors to the Eclipse Foundation
4+
#
5+
# See the NOTICE file(s) distributed with this work for additional
6+
# information regarding copyright ownership.
7+
#
8+
# This program and the accompanying materials are made available under the
9+
# terms of the Apache License Version 2.0 which is available at
10+
# https://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# SPDX-License-Identifier: Apache-2.0
13+
# *******************************************************************************
14+
15+
.. _itf_concepts:
16+
17+
Concepts
18+
========
19+
20+
Architecture, design decisions, and explanatory material for ITF.
21+
22+
.. toctree::
23+
:maxdepth: 1
24+
25+
architecture

‎docs/concepts/itf_architecture.puml‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
@startuml ITF Architecture
2+
!pragma layout smetana
3+
skinparam componentStyle rectangle
4+
skinparam defaultFontName sans-serif
5+
skinparam linetype ortho
6+
7+
' ── Test layer ────────────────────────────────────────────────────────────────
8+
package "Test Code" {
9+
component "test_*.py\n(pytest test function)" as TestFn
10+
component "@requires_capabilities\n(optional guard)" as RC
11+
}
12+
13+
' ── ITF Core ──────────────────────────────────────────────────────────────────
14+
package "ITF Core (score.itf.plugins.core)" {
15+
component "target fixture" as TargetFx
16+
component "target_init fixture\n(plugin hook)" as TargetInit
17+
}
18+
19+
' ── Target abstraction ────────────────────────────────────────────────────────
20+
package "Target Abstraction" {
21+
component "Target (base class)\n──────────────────\nexecute(cmd)\nupload(src, dst)\ndownload(src, dst)\nrestart()\nhas_capability(cap)\nget_capabilities() : Set[str]" as Target
22+
}
23+
24+
' ── Plugin implementations ────────────────────────────────────────────────────
25+
package "Plugins" {
26+
component "Docker Plugin\n──────────\ncapabilities:\nexec | file_transfer\nrestart" as DockerPlugin
27+
28+
component "QEMU Plugin\n──────────\ncapabilities:\nssh | sftp | exec\nfile_transfer | restart" as QemuPlugin
29+
30+
component "DLT Plugin\n──────────\nDltWindow\ndlt_config fixture" as DltPlugin
31+
}
32+
33+
' ── Infrastructure ────────────────────────────────────────────────────────────
34+
package "Infrastructure" {
35+
database "Docker Daemon" as DockerInfra
36+
database "QEMU VM" as QemuInfra
37+
}
38+
39+
' ── Bazel build layer ─────────────────────────────────────────────────────────
40+
package "Bazel Build" {
41+
component "py_itf_test\n(symbolic macro)" as PyItfTest
42+
component "py_itf_plugin\n(rule)" as PyItfPlugin
43+
}
44+
45+
' ── Relationships ─────────────────────────────────────────────────────────────
46+
TestFn -down-> TargetFx : receives
47+
TestFn .right.> RC : guarded by
48+
TargetFx -down-> TargetInit : delegates to
49+
TargetInit -down-> Target : yields instance of
50+
DockerPlugin -up--|> Target : implements
51+
QemuPlugin -up--|> Target : implements
52+
53+
DockerPlugin -down-> DockerInfra : manages lifecycle
54+
QemuPlugin -down-> QemuInfra : manages lifecycle
55+
56+
TestFn ..> DltPlugin : optional\nDltWindow usage
57+
58+
PyItfTest -right-> PyItfPlugin : composes plugins
59+
PyItfPlugin -up-> TargetInit : provides\ntarget_init fixture
60+
61+
@enduml

‎docs/how-to/get_started.md‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
# Get Started
2+
3+
This guide walks you through setting up ITF in a new Bazel workspace from scratch.
4+
5+
## Prerequisites
6+
7+
- [Bazel](https://bazel.build/install) 7.x or later
8+
- Docker (for Docker-based tests)
9+
- Python 3.12+
10+
11+
## 1. Add ITF to your workspace
12+
13+
In your `MODULE.bazel`, declare the dependency using the latest published
14+
registry version:
15+
16+
```starlark
17+
bazel_dep(name = "score_itf", version = "0.2.0")
18+
```
19+
20+
To get unreleased fixes from the main branch, add a `git_override` directly
21+
after the `bazel_dep`:
22+
23+
```starlark
24+
git_override(
25+
module_name = "score_itf",
26+
remote = "https://github.com/eclipse-score/itf.git",
27+
commit = "<COMMIT_HASH>",
28+
)
29+
```
30+
31+
Replace `<COMMIT_HASH>` with the full SHA of the desired commit from the
32+
[score_itf repository](https://github.com/eclipse-score/itf).
33+
34+
## 2. Configure `.bazelrc`
35+
36+
Add the S-CORE Bazel registry so Bazel can resolve the `score_itf` module:
37+
38+
```
39+
common --registry=https://raw.githubusercontent.com/eclipse-score/bazel_registry/main/
40+
common --registry=https://bcr.bazel.build
41+
```
42+
43+
If you also want to build the ITF documentation locally (optional), add the
44+
Java configuration required by PlantUML:
45+
46+
```
47+
build --java_language_version=17
48+
build --java_runtime_version=remotejdk_17
49+
build --tool_java_language_version=17
50+
build --tool_java_runtime_version=remotejdk_17
51+
```
52+
53+
## 3. Write your first test
54+
55+
Create a test file `test_hello.py`:
56+
57+
```python
58+
def test_hello(target):
59+
exit_code, output = target.execute("echo 'Hello from target!'")
60+
assert exit_code == 0
61+
assert b"Hello from target!" in output
62+
```
63+
64+
Create a `BUILD` file in the same directory:
65+
66+
```starlark
67+
load("@score_itf//:defs.bzl", "py_itf_test")
68+
69+
py_itf_test(
70+
name = "test_hello",
71+
srcs = ["test_hello.py"],
72+
args = ["--docker-image=ubuntu:24.04"],
73+
plugins = ["@score_itf//score/itf/plugins:docker_plugin"],
74+
)
75+
```
76+
77+
> **Note:** All `load()` calls and plugin labels must use the `@score_itf`
78+
> prefix. Using `//:defs.bzl` without the prefix would look for that file
79+
> in your own workspace and fail with `no such file`.
80+
81+
## 4. Run the test
82+
83+
```bash
84+
bazel test //path/to:test_hello
85+
```
86+
87+
To see full test output:
88+
89+
```bash
90+
bazel test //path/to:test_hello --test_output=all
91+
```
92+
93+
## 5. Link tests to requirements
94+
95+
ITF integrates with the S-CORE docs-as-code traceability system. The
96+
`@add_test_properties` decorator writes requirement links and test
97+
classification metadata into the JUnit XML report, which Sphinx can then
98+
pick up to create bidirectional traceability between test results and
99+
requirements.
100+
101+
```python
102+
from attribute_plugin import add_test_properties
103+
104+
@add_test_properties(
105+
fully_verifies=["REQ-001", "REQ-002"],
106+
test_type="requirements-based",
107+
derivation_technique="requirements-analysis",
108+
)
109+
def test_hello(target):
110+
exit_code, output = target.execute("echo 'Hello from target!'")
111+
assert exit_code == 0
112+
assert b"Hello from target!" in output
113+
```
114+
115+
Add `attribute_plugin` to the `plugins` list in your `BUILD` file:
116+
117+
```starlark
118+
py_itf_test(
119+
name = "test_hello",
120+
srcs = ["test_hello.py"],
121+
args = ["--docker-image=ubuntu:24.04"],
122+
plugins = [
123+
"@score_itf//score/itf/plugins:docker_plugin",
124+
"@score_itf//score/itf/plugins:attribute_plugin",
125+
],
126+
)
127+
```
128+
129+
## Next steps
130+
131+
- **Write more tests**: See [Write Tests](write_tests.md) for Docker, QEMU,
132+
and DLT examples.
133+
- **Use plugins**: See [Using Plugins](plugins.md) for configuring and
134+
combining built-in plugins.
135+
- **Understand the design**: See the [Concepts](../concepts/index.rst)
136+
section for the architecture and plugin system.
137+
138+
## For ITF contributors
139+
140+
Build this documentation site locally:
141+
142+
```bash
143+
bazel run //:docs
144+
```
145+
146+
Open `_build/index.html` in your browser.

0 commit comments

Comments
 (0)