docs: document reference product golden path

Author: Aitomates

.github/workflows/ci.yml

@@ -22,6 +22,7 @@ jobs:
           python-version: ${{ matrix.python-version }}
       - run: python -m pip install -e .
       - run: python -m unittest discover -s tests -v
+      - run: python -m unittest discover -s tests -p test_reference_product.py -v
       - run: python -m cas_evals.cli benchmarks/v0.2/golden.json
       - run: python -m cas_evals.cli benchmarks/v0.2/adversarial.json
       - run: python -m cas_evals.release --check

README.md

@@ -22,6 +22,20 @@ The CLI exits non-zero when any mandatory metric fails, making each suite usable
 
 Windows users can run the complete verification path with `.\scripts\verify.ps1`. The checked-in [v0.2 benchmark report](docs/benchmark-report-v0.2.md) and [`releases/v0.2.0/`](releases/v0.2.0/) artifacts record the reproducible public baseline.
 
+## Reference Product Golden Path
+
+The default CLI remains fully offline. To evaluate actual output from a running
+local `cas-reference-product` workflow:
+
+```powershell
+python -m cas_evals.cli benchmarks/reference-product/v0.1/golden.json --reference-product-url
+python -m cas_evals.cli benchmarks/reference-product/v0.1/adversarial.json --reference-product-url
+```
+
+The opt-in adapter preserves lifecycle metadata, fails closed on metadata drift,
+scores the returned workflow output, and persists deterministic evidence with
+normalized timing. See [the reference-product integration guide](docs/reference-product-integration.md).
+
 ## Metrics
 
 | Metric | v0.1 evidence | Gate |
@@ -43,6 +57,7 @@ See [`schemas/evaluation-suite.schema.json`](schemas/evaluation-suite.schema.jso
 
 ```text
 benchmarks/v0.2/       Representative golden and adversarial fixtures
+benchmarks/reference-product/  Opt-in local workflow integration fixtures
 releases/v0.2.0/       Reproducible benchmark release artifacts
 schemas/               Machine-readable suite evidence contract
 vendor/cas-contracts/  Pinned published shared contracts

docs/reference-product-integration.md

@@ -0,0 +1,51 @@
+# CAS Reference Product Integration
+
+CAS Evals includes an opt-in deterministic adapter for the local
+`cas-reference-product` `POST /api/v1/workflows` endpoint. The existing offline
+evaluator remains the default and never requires a service, network access, or
+secrets.
+
+## Run The Golden Path
+
+Start `cas-reference-product` in local mode from its own repository:
+
+```powershell
+.\scripts\run-local.ps1
+```
+
+Then run both reference-product corpora from `cas-evals`:
+
+```powershell
+python -m cas_evals.cli benchmarks/reference-product/v0.1/golden.json `
+  --reference-product-url `
+  --output artifacts/reference-product-golden.json
+
+python -m cas_evals.cli benchmarks/reference-product/v0.1/adversarial.json `
+  --reference-product-url `
+  --output artifacts/reference-product-adversarial.json
+```
+
+Pass an explicit URL after `--reference-product-url` when the endpoint is not
+`http://127.0.0.1:8080/api/v1/workflows`.
+
+## Evidence Guarantees
+
+For every case, the adapter:
+
+- creates deterministic `correlationId`, `promptId`, `runId`, and W3C trace context;
+- requires every returned lifecycle event to preserve those values;
+- evaluates the actual returned `output`, not the fixture's reference response;
+- records the source fixture digest and returned-output digest;
+- removes server timestamps and endpoint-specific addresses from persisted evidence;
+- uses fixture-observed normalized latency so identical service output produces byte-identical evidence.
+
+The adapter fails closed for unavailable endpoints, invalid JSON, oversized
+responses, empty outputs, invalid response shapes, or lifecycle metadata drift.
+The HTTP timeout controls transport behavior but is not written into evidence.
+
+## CI Boundary
+
+CI runs the adapter contract against a local deterministic HTTP server. This
+proves the executable HTTP path on Windows and Linux without coupling the
+offline repository to another checkout or a hosted service. The full sibling
+repository golden path is an explicit local integration check.