Skip to content

Commit c1a3571

Browse files
add 11
1 parent 859a97d commit c1a3571

6 files changed

Lines changed: 74 additions & 19 deletions

File tree

docs/explanation/04-testing-infrastructure.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -222,13 +222,13 @@ The infrastructure challenge is therefore layered. Cloud-based benchmark targets
222222

223223
*Infrastructure for collecting, aggregating, and presenting test results as verification evidence across S-CORE.*
224224

225-
Test results are already visible in several places, but they do not yet form one consistent project-wide reporting layer. GitHub Actions exposes outcomes per pipeline run, release flows can aggregate and attach selected test and coverage artifacts, and some repositories already publish dashboard-style views for traceability or unit-test summaries. `reference_integration` also plays an important role in collecting higher-level evidence once modules are assembled and exercised together. The infrastructure direction is therefore visible: reporting should turn execution results into durable evidence that can be reviewed per run, per release, and eventually across repositories. **Biggest gap**: no centralized project-wide dashboard or durable cross-repository reporting model yet spans all of S-CORE.
225+
Test results are already visible in several places, but they do not yet form one consistent project-wide reporting layer. GitHub Actions exposes outcomes per pipeline run, release flows can aggregate and attach selected test and coverage artifacts, and some repositories already publish dashboard-style views for traceability or unit-test summaries. `reference_integration` is one of the main places where higher-level evidence is collected once modules are assembled and exercised together; [chapter 11](11-reference-integration.md) explains why that snapshot boundary matters. The infrastructure direction is therefore visible: reporting should turn execution results into durable evidence that can be reviewed per run, per release, and eventually across repositories. **Biggest gap**: no centralized project-wide dashboard or durable cross-repository reporting model yet spans all of S-CORE.
226226

227227
### 4.4.1 Result Aggregation 🟠
228228

229229
*Infrastructure aggregating test results across CI pipeline runs.*
230230

231-
Aggregation already exists in pieces. CI runs produce artifacts, and release-oriented flows can combine selected outputs into something closer to a reusable evidence package. For higher integration levels, `reference_integration` is an especially important aggregation point because it collects results after cross-repository assembly and scenario execution. To stay meaningful, those aggregated results should be keyed to a concrete `known_good` manifest or record rather than to a vague notion of "current main". The fast integration subset can provide early feedback against a candidate manifest, but the deeper post-merge or scheduled suite is what should advance the stored known-good baseline. **Biggest gap**: aggregation works for some release flows, but continuous project-wide aggregation across repositories is still incomplete.
231+
Aggregation already exists in pieces. CI runs produce artifacts, and release-oriented flows can combine selected outputs into something closer to a reusable evidence package. For higher integration levels, `reference_integration` is an especially important aggregation point because it collects results after cross-repository assembly and scenario execution. To stay meaningful, those aggregated results should be keyed to a concrete `known_good` manifest or record rather than to a vague notion of "current main". The fast integration subset can provide early feedback against a candidate manifest, but the deeper post-merge or scheduled suite is what should advance the stored known-good baseline. [Chapter 11](11-reference-integration.md) describes the control model behind that boundary. **Biggest gap**: aggregation works for some release flows, but continuous project-wide aggregation across repositories is still incomplete.
232232

233233
### 4.4.2 Test Dashboards 🔴
234234

docs/explanation/07-automation-integration.md

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -161,6 +161,7 @@ Where repositories use dependency lock files, the CI perspective should be enfor
161161

162162
- Cross-repository automation needs more than triggers; it also needs a repeatable way to hand off artifacts, SBOMs, and verification evidence between jobs or repositories.
163163
- That handover may include the `known_good` manifest or identifier, module references resolved through `bazel_registry`, release assets, and the test outputs produced by `reference_integration`.
164+
- [Chapter 11](11-reference-integration.md) defines the snapshot boundary that makes those artifacts meaningful.
164165
- **Biggest gap**: no shared artifact handover model exists for multi-repository automation scenarios.
165166

166167
### 7.3.4 Known-Good Promotion 🔴
@@ -169,11 +170,9 @@ Where repositories use dependency lock files, the CI perspective should be enfor
169170

170171
`known_good` gives CI a stable unit of promotion. Module repositories can continue to publish releases, registry entries, or candidate revisions independently, but cross-repository automation needs one place where those inputs are assembled into a candidate stack, executed together, and either promoted or rejected with clear feedback. The important nuance is that CI does not treat `known_good` as a byproduct of Bazel resolution. It treats it as the higher-level control file that selects the candidate stack and can also carry automation metadata such as which branch should be followed for automatic updates.
171172

172-
Assuming the still-undecided Option 2 direction currently under discussion, `reference_integration` would own that promotion gate. For a pull request in one repository, CI would build a candidate manifest from the changed repository ref together with the last known-good refs for the other participating repositories, generate the Bazel-facing inputs needed for the integrated workspace from that manifest, and run the fast integration subset for early feedback. After merges, or on a schedule, `reference_integration` would build a fuller manifest from the latest eligible branches, run the deeper suite, and only on success update the stored `known_good` record. If the final scope of `reference_integration` is narrowed later, the same promotion pattern still makes sense, but the meaning of `known_good` must be reduced to the checks that are actually re-executed centrally.
173+
The detailed ownership and evidence story now lives in [chapter 11](11-reference-integration.md). This section keeps the CI orchestration perspective: for a pull request in one repository, CI needs a candidate manifest from the changed repository ref together with the last known-good refs for the other participating repositories, plus a repeatable path from that manifest to the integrated workspace inputs and results.
173174

174-
That also gives the handover model a natural key: artifacts, SBOMs, reports, and logs should be traceable to the `known_good` identifier or manifest hash they were produced for rather than only to an individual repository run. **Biggest gap**: S-CORE has no standardized candidate-manifest construction, `known_good` promotion workflow, ownership model, or result schema for cross-repository CI.
175-
176-
The workflow shape described here follows the distributed-monolith integration model in [DR-002-Infra](https://eclipse-score.github.io/score/main/design_decisions/DR-002-infra.html), while the stronger central ownership assumed in some sentences still depends on the unresolved Option 2 versus lighter-scope discussion in [DR-008-Int](https://github.com/qorix-group/score/blob/da4ea900f1eece5c8e795697d71e277446dca84e/docs/design_decisions/DR-008-int.rst?plain=1).
175+
The workflow shape described here follows the distributed-monolith integration model in [DR-002-Infra](https://eclipse-score.github.io/score/main/design_decisions/DR-002-infra.html), and the selected DR-008 Option 4 model keeps the stronger integration contract while validating module repositories against the exact dependency set resolved by the integrated baseline.
177176

178177
## 7.4 Secrets Management 🔴
179178

docs/explanation/09-documentation-infrastructure.md

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ Publishing should be an explicit, reproducible stage of the docs pipeline rather
7474
*Establishing reliable links across repository boundaries and release versions.*
7575

7676
- Stable links are required if documentation, code, requirements, and release artifacts live in different repositories.
77-
- For integrated views produced from `reference_integration`, those links should resolve within one explicit `known_good` snapshot rather than silently mixing repository heads.
77+
- For integrated views produced from `reference_integration`, those links should resolve within one explicit `known_good` snapshot rather than silently mixing repository heads. [Chapter 11](11-reference-integration.md) explains why that snapshot boundary matters.
7878
- **Biggest gap**: no agreed cross-repository linking strategy exists for versioned and unversioned documentation content.
7979

8080
### 9.3.2 Shared Navigation and Discovery 🔴
@@ -118,8 +118,6 @@ For S-CORE, the practical value is that test evidence produced in [chapter 4](04
118118

119119
Cross-repository documentation becomes ambiguous unless readers can tell which combination of component revisions it describes. The `known_good` concept solves that by giving integrated documentation, verification summaries, and traceability views one shared snapshot identifier. A single documentation build in `reference_integration` should therefore describe one concrete `known_good` manifest or record, not an unspecified mix of module heads.
120120

121-
Under the currently assumed but still-undecided Option 2 model, this identifier is also what allows the project to claim that integrated docs and integrated evidence were generated from the same centrally validated stack. If `reference_integration` later ends up with a lighter scope, the documentation benefit remains, but the traceability story has to distinguish more clearly between centrally generated pages and evidence linked in from module repositories.
121+
The important architectural point is that links, release notes, dashboards, and archived evidence should all resolve back to the same `known_good` identifier. [Chapter 11](11-reference-integration.md) owns the repository-level explanation of that snapshot contract. **Biggest gap**: S-CORE does not yet have a documented rule for binding integrated documentation and traceability artifacts to one explicit cross-repository snapshot.
122122

123-
The important architectural point is that links, release notes, dashboards, and archived evidence should all resolve back to the same `known_good` identifier. **Biggest gap**: S-CORE does not yet have a documented rule for binding integrated documentation and traceability artifacts to one explicit cross-repository snapshot.
124-
125-
This snapshot-oriented documentation view depends on the same integration model described in [DR-002-Infra](https://eclipse-score.github.io/score/main/design_decisions/DR-002-infra.html) and on the still-unsettled `reference_integration` scope discussion in [DR-008-Int](https://github.com/qorix-group/score/blob/da4ea900f1eece5c8e795697d71e277446dca84e/docs/design_decisions/DR-008-int.rst?plain=1).
123+
This snapshot-oriented documentation view depends on the same integration model described in [DR-002-Infra](https://eclipse-score.github.io/score/main/design_decisions/DR-002-infra.html) and on the selected `reference_integration` scope in [DR-008-Int](https://github.com/qorix-group/score/blob/da4ea900f1eece5c8e795697d71e277446dca84e/docs/design_decisions/DR-008-int.rst?plain=1).
Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# 11 Reference Integration 🟡
2+
3+
:::{tip} Looking for practical guides?
4+
This chapter is part of the infrastructure landscape assessment. For step-by-step how-tos and quick references, see the [How-to Guides](../how-to/index.md).
5+
:::
6+
7+
*Cross-repository integration workspace for validating S-CORE modules as one coherent stack.*
8+
9+
`reference_integration` is the place where S-CORE turns multiple module repositories into one validated integration context. It is not a replacement for module-local CI. Instead, it provides the shared workspace, candidate-stack promotion model, and consolidated evidence boundary that module repositories cannot define on their own. The current DR-008 operating model is stable `known_good` with module-scoped validation.
10+
11+
That distinction matters because the repository sits at the seam between independent module ownership and project-level release confidence. Modules still own their unit tests, static analysis, and repository-local quality gates. `reference_integration` owns the cross-repository view: it assembles a concrete candidate stack, validates that stack against a shared dependency set, and records the outcome as a durable integration snapshot.
12+
13+
**Biggest gap**: the integration role is now defined, but the orchestration, evidence consolidation, and module-scoped validation plumbing still need to be implemented consistently across repositories.
14+
15+
## 11.1 Integration Workspace 🟡
16+
17+
*A shared Bazel workspace for exercising multiple S-CORE modules together.*
18+
19+
The repository's first job is to make a multi-module checkout usable as one build and test environment. That workspace model is what exposes cross-repository dependency problems, label mismatches, unsupported toolchain combinations, and integration regressions that would stay hidden if every module only validated itself in isolation.
20+
21+
The workspace is intentionally broader than a simple aggregation of repository checkouts. It includes the glue needed to make the integrated stack buildable and debuggable: repository overrides, shared Bazel configuration, runnable examples, and structured entry points for the supported integration scenarios. The result is a controlled integration baseline rather than a vague collection of latest heads.
22+
23+
**Biggest gap**: workspace assembly and update mechanics are still more operationally complex than they should be for contributors who only want to validate a candidate stack.
24+
25+
## 11.2 Known-Good Promotion 🟡
26+
27+
*A concrete snapshot model for deciding which component revisions belong together.*
28+
29+
The integration unit is `known_good`, not an arbitrary combination of current branches. That makes the stack reproducible and reviewable: a promotion either selects one explicit set of module revisions or it does not. The same identifier can then drive CI, documentation, traceability views, and artifact retention without mixing unrelated repository heads.
30+
31+
This snapshot model is the architectural bridge between module-level release flow and project-level integration flow. Module repositories can keep moving independently, but `reference_integration` defines which exact revisions are accepted as one coherent stack for a given point in time. When the snapshot changes, the integrated evidence changes with it.
32+
33+
**Biggest gap**: S-CORE still lacks one shared rule set for how candidate stacks are assembled, promoted, and tagged across all participating repositories.
34+
35+
## 11.3 Integrated Evidence 🟠
36+
37+
*Collecting verification, documentation, and release evidence from the same validated stack.*
38+
39+
The value of the integration repository is not just that it can build a combined workspace. It also becomes the place where higher-level evidence is attached to one concrete stack identity. That includes cross-repository test results, documentation builds for release scenarios, dependency-resolution outputs, and the metadata that makes those results auditable later.
40+
41+
The important constraint is consistency. Evidence that is produced for one `known_good` snapshot should remain traceable to that snapshot, not to the moving target of "whatever was on main when the job ran". That is what makes the repository useful for release confidence, not just for ad hoc debugging.
42+
43+
For documentation in particular, the repository gives S-CORE a place to describe integrated views as integrated views. The explanation chapters in this site treat `reference_integration` as the canonical place for that cross-repository snapshot discussion, while the documentation chapter keeps the consumer-facing implications of the same model.
44+
45+
**Biggest gap**: S-CORE does not yet have one uniform schema for integrated evidence, so results remain spread across CI artifacts, repository docs, and release outputs.
46+
47+
## 11.4 Operating Model 🟡
48+
49+
*Stable `known_good` with module-scoped validation.*
50+
51+
The selected model keeps `reference_integration` focused on the integrated stack while leaving module-owned quality checks in the module repositories. In practice, that means `reference_integration` promotes one explicit stack, runs the integration-scoped checks that belong to the shared workspace, and validates the module repositories against the dependency set resolved by that stack.
52+
53+
This gives S-CORE a strong integration contract without forcing every verification concern into the central repository. The important consequence is that the integrated baseline remains the source of truth for cross-repository evidence, but the diagnostic depth of module-local validation stays close to the owning teams.
54+
55+
The remaining work is operational rather than architectural: standardized handoff of resolved dependency versions, reliable consolidation of reports and artifacts, and clear handling of temporary validation state so release evidence stays auditable.

docs/explanation/decisions.md

Lines changed: 5 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -42,16 +42,13 @@ Key architectural decisions that shape the S-CORE infrastructure. Each entry sum
4242

4343
---
4444

45-
## DR-008-Int — `reference_integration` Scope (Open)
45+
## DR-008-Int — `reference_integration` Scope
4646

47-
**Status**: Not yet decided.
47+
**Status**: Decided. Option 4 was selected.
4848

49-
**Problem**: Two options are under discussion for the scope of `reference_integration`:
49+
**Decision**: Use a stable `known_good` baseline with module-scoped validation. `reference_integration` promotes one explicit integrated stack, runs the integration-scoped checks on that stack, and validates module repositories against the dependency set resolved by the integrated baseline.
5050

51-
- **Option 2** (stronger central ownership): `reference_integration` rebuilds and re-tests the full integrated stack for every `known_good` promotion. All cross-repository verification evidence is produced centrally.
52-
- **Lighter scope**: `reference_integration` assembles and promotes `known_good` but delegates more verification to individual component repositories. Some evidence stays per-module.
53-
54-
**Effect on docs**: Several sections in [Chapter 3](03-build-infrastructure.md) and [Chapter 7](07-automation-integration.md) note where their architecture description assumes Option 2 and where it would change under a lighter scope.
51+
**Effect on docs**: Chapter [7](07-automation-integration.md) describes the CI promotion flow against the selected baseline. Chapter [11](11-reference-integration.md) is the canonical explanation of the integration workspace, promotion model, and evidence boundary.
5552

5653
**Full DR**: [DR-008-Int](https://github.com/qorix-group/score/blob/da4ea900f1eece5c8e795697d71e277446dca84e/docs/design_decisions/DR-008-int.rst?plain=1)
5754

@@ -66,6 +63,6 @@ This section was formerly part of [Chapter 3 § 3.2.7](03-build-infrastructure.m
6663
- A **lock file** (`MODULE.bazel.lock`, `uv.lock`) captures the resolved dependency graph that a concrete Bazel workspace consumes.
6764
- A **`known_good` manifest** is the curated integration selection from which Bazel-facing inputs can be generated. It can also carry automation metadata such as timestamps, suite identity, and which branch should be followed for automated CI refreshes.
6865

69-
Regardless of whether the project eventually adopts Option 2 or a lighter `reference_integration` scope, `known_good` should be version-controlled, diffable, reproducible, and clearly distinct from generated Bazel lock data. The concrete file format matters less than that behavior.
66+
`known_good` should be version-controlled, diffable, reproducible, and clearly distinct from generated Bazel lock data. The concrete file format matters less than that behavior.
7067

7168
**Decision background**: [DR-001-Strat](https://eclipse-score.github.io/score/main/design_decisions/DR-001-strat.html) · [DR-002-Infra](https://eclipse-score.github.io/score/main/design_decisions/DR-002-infra.html) · [DR-008-Int](https://github.com/qorix-group/score/blob/da4ea900f1eece5c8e795697d71e277446dca84e/docs/design_decisions/DR-008-int.rst?plain=1)

0 commit comments

Comments
 (0)