Skip to content

Commit 153e7e8

Browse files
docs: frame drift tracer as a workaround
1 parent 3b096b0 commit 153e7e8

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

docs-website/docs/development/tracing.mdx

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -358,11 +358,11 @@ Here’s what the resulting log would look like when a pipeline is run:
358358
<ClickableImage src="/img/55c3d5c84282d726c95fb3350ec36be49a354edca8a6164f5dffdab7121cec58-image_2.png" alt="Console output showing Haystack pipeline execution with DEBUG level tracing logs including component names, types, and input/output specifications" />
359359

360360

361-
## Behavioral Drift Monitoring with a Custom Tracer
361+
## Cross-Run Behavioral Drift Monitoring with a Custom Tracer
362362

363-
Haystack's `Tracer` interface can be used for more than routing spans to a backend — it can also detect **behavioral drift** across pipeline runs. This is useful when your pipeline uses a retrieval-augmented or context-compression component and you want to know whether the agent's effective vocabulary is shifting between sessions.
363+
Haystack's `Tracer` interface can be used for more than routing spans to a backend — it can also detect **behavioral drift** across pipeline runs. This is a workaround for users who want extra observability today without adding a first-class context-compaction event to Haystack itself.
364364

365-
The example below implements a lightweight `DriftMonitorTracer` that tracks which domain-specific terms (the "ghost lexicon") appear in the first pipeline run but disappear in later runs. A Ghost Consistency Score (GCS) below 0.40 typically signals that the pipeline is losing context-critical vocabulary.
365+
The example below implements a lightweight `DriftMonitorTracer` that tracks which domain-specific terms (the "ghost lexicon") appear in the first pipeline run but disappear in later runs. A Ghost Consistency Score (GCS) below 0.40 typically signals that the pipeline is losing context-critical vocabulary across runs.
366366

367367
```python
368368
import contextlib
@@ -521,5 +521,5 @@ if report["alert"]:
521521
This pattern requires no changes to Haystack internals. The `Tracer` interface is the only extension point needed. For production use, extend `_on_span_finished` to maintain a per-run rolling window and compare against a configurable baseline depth rather than only the first run.
522522

523523
:::note
524-
This addresses the behavioral-drift monitoring use case from [#10971](https://github.com/deepset-ai/haystack/issues/10971) using the existing `Tracer` interface — no new hooks required.
524+
This addresses part of the behavioral-drift monitoring use case from [#10971](https://github.com/deepset-ai/haystack/issues/10971) using the existing `Tracer` interface. It is not a first-class boundary event for summarizers or truncators; it is an explicit workaround based on traced inputs and outputs.
525525
:::

0 commit comments

Comments
 (0)