Skip to content

Commit 22d290f

Browse files
docs: clarify golden artifacts purpose and testing limitations
1 parent ffbbfc7 commit 22d290f

1 file changed

Lines changed: 11 additions & 3 deletions

File tree

docs/CONTRIBUTING.md

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -162,7 +162,9 @@ npm run test:spelling
162162

163163
### Golden artifacts (regression testing)
164164

165-
Golden artifacts are reference outputs stored in `fixtures/golden/` that serve as snapshots for regression testing. They ensure pipeline changes don't accidentally alter outputs.
165+
Golden artifacts are reference outputs stored in `fixtures/golden/` that serve as snapshots for regression testing. They are a narrow contract guard for published artifacts, not a proof that the full pipeline is semantically correct.
166+
167+
The check intentionally compares sanitized, deterministic output only. Volatile live fields such as timestamps, stars, issue counters, and other environment-dependent values are normalized or excluded so the snapshot highlights contract drift instead of operational noise.
166168

167169
**Workflow when modifying pipeline code:**
168170

@@ -192,10 +194,16 @@ Golden artifacts are reference outputs stored in `fixtures/golden/` that serve a
192194

193195
**What they test:**
194196

195-
- Pipeline produces consistent outputs for the same inputs
196-
- No accidental regressions in data structure or content
197+
- Published artifact shape stays stable after deterministic sanitization
198+
- No accidental regressions in committed contract-level output
197199
- Contract changes are explicit and reviewable in PRs
198200

201+
**What they do not test:**
202+
203+
- End-to-end semantic correctness of the full pipeline
204+
- Live repository state, current stars, or other volatile upstream data
205+
- Non-deterministic outputs such as `skipped_modules.json`
206+
199207
## Helpful references
200208

201209
- [`docs/architecture.md`](architecture.md) – current pipeline topology.

0 commit comments

Comments
 (0)