Commit a497306
fix(e2e): fix all hermetic CI failures
Four distinct root causes addressed:
1. Empty body mismatch (HuggingFace, all GET requests)
The legacy format stored zero-length request bodies as
{ bodyEncoding: "utf8", body: "" }. The converter was producing
{ kind: "text", value: "" }, but seinfeld's encodeBody() returns
{ kind: "empty" } for incoming zero-length requests. deepEqual
comparison failed kind !== kind.
Fix: converter now returns { kind: "empty" } for empty text bodies;
seinfeld's bodyEqual normalizes { kind: "text", value: "" } to
{ kind: "empty" } before comparison.
2. cassetteEngaged wrong file extension (OpenAI assertions)
openai-instrumentation/assertions.ts:618 checked existsSync for
${snapshotName}.json but cassette files are *.cassette.json.
cassetteEngaged was always false, causing the strict
expect(span?.output).toBeUndefined() branch to run during cassette
replay (which always has a defined output from buffered SSE).
Fix: change extension to .cassette.json.
3. AI SDK v5/v6 Responses API body drift
Cassettes were recorded with older AI SDK minor versions. Newer
versions (5.0.82, 6.0.1) add Responses API default fields like
store, background, truncation, reasoning, service_tier, metadata,
etc. deepEqual on the full request body failed for 3 of 7 entries.
Fix: add comprehensive ignoreBodyFields for known Responses API
drift fields to the ai-sdk-instrumentation and ai-sdk-otel-export
filter specs.
4. Cassette SSE chunking and prettier formatting
All 23 cassettes re-converted from original git sources to pick up
both the empty-body fix and the previously-fixed SSE chunk splitting.
Prettier applied to all cassette JSON files.
Pre-existing failures NOT addressed (require re-recording with API keys):
- Mistral: no cassette directory
- Google ADK: stale snapshots
- Cohere 7.20.0+/8.0.0: no cassettes for newer versions
- js-provider-tests: Anthropic streaming LLM output flakiness
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>1 parent 0f9eccc commit a497306
29 files changed
Lines changed: 7023 additions & 19919 deletions
File tree
- dev-packages/seinfeld
- scripts
- src/matcher
- e2e
- helpers
- scenarios
- ai-sdk-instrumentation/__cassettes__
- ai-sdk-otel-export/__cassettes__
- anthropic-instrumentation/__cassettes__
- cohere-instrumentation/__cassettes__
- groq-instrumentation/__cassettes__
- huggingface-instrumentation/__cassettes__
- openai-instrumentation
- __cassettes__
- openrouter-agent-instrumentation/__cassettes__
- openrouter-instrumentation/__cassettes__
- wrap-langchain-js-traces/__cassettes__
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
199 | 199 | | |
200 | 200 | | |
201 | 201 | | |
202 | | - | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
203 | 205 | | |
204 | 206 | | |
205 | 207 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
33 | 41 | | |
34 | | - | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
35 | 48 | | |
36 | 49 | | |
37 | 50 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
13 | 14 | | |
14 | 15 | | |
15 | 16 | | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
16 | 45 | | |
17 | 46 | | |
18 | 47 | | |
| |||
0 commit comments