Version-aware OTel GenAI semconv normalization

[krisztianfekete]

Problem or motivation

The OTel GenAI semantic conventions churn across releases. Our extraction is written against the latest attribute names, so traces emitted against older or newer spec versions can silently under-extract. We currently ignore the schemaUrl that identifies which version a span was emitted or normalized to.

Proposed solution

Make extraction version-aware, probably via schema_url as a hint:

1. Capture scopeSpans.schemaUrl at ingest
2. Resolve gen_ai.* attributes through a small alias table (canonical/newest key + older aliases) so latest-name extraction still reads older traces
3. Use it in GenAIExtractor to make extraction more robust, and expose the resolved semconv version for run metadata

We should degrade gracefully when schemaUrl is missing.

Alternatives considered

No response

Additional context

No response

Human confirmation

• I am a human (not a bot, agent, or AI) filing this issue.

[AaronProbha18]

@krisztianfekete
I'd like to work on this issue.

Plan

The main goal is to make GenAI attribute extraction resilient across OpenTelemetry semantic convention versions instead of depending solely on the latest attribute names. Right now, traces can lose information because schemaUrl is ignored during ingestion and renamed gen_ai.* attributes are never normalized. My approach is to first identify the run metadata model so the resolved semantic convention version can be persisted, then add schema URL propagation across every ingestion path, introduce a centralized alias resolution layer, and finally extend extraction and metadata generation to consume that information. A key design choice is to normalize attributes once during ingestion so existing extraction code can continue reading canonical keys without widespread changes. Missing or malformed schemaUrl should degrade cleanly to an "unknown" version without affecting span processing, while canonical attributes should always take precedence over aliases to avoid overwriting valid data.

Changes

src/agentevals/loader/base.py will be updated to carry schema_url on Span, allowing scope-level schema information to travel alongside parsed spans. src/agentevals/trace_attrs.py will define the new schema URL constant and maintain the canonical-to-alias mapping for historical gen_ai.* attribute names. In src/agentevals/extraction.py, I'll add alias normalization and semantic convention version resolution so downstream extraction logic continues using canonical lookups unchanged.

src/agentevals/loader/otlp.py, src/agentevals/api/otlp_processing.py, and src/agentevals/streaming/processor.py all need parallel updates to capture schemaUrl from their respective ingestion flows and apply the shared normalization consistently. Once the run metadata model is confirmed under src/agentevals/storage/ (or the corresponding API model), I'll extend it with the resolved semantic convention version and add any required migration. I'll also expand tests/test_extraction.py (or the equivalent test module), update any span fixtures affected by the new field, and document the compatibility behavior in docs/otel-compatibility.md.

• Add schema_url propagation across all ingestion paths.
• Implement shared GenAI alias normalization and semantic version resolution.
• Expose the resolved semantic convention version in run metadata.
• Add unit and integration coverage for aliases, schema handling, and metadata persistence.
• Update compatibility documentation and related comments.

[krisztianfekete]

@AaronProbha18, thanks for looking into this!

Couple of comments:

• Resolve aliases at read time, don't rewrite attributes during ingestion. agentevals is a "trace inspection tool", so I'd rather keep the stored spans exactly as they came in and only fall back to older attribute names at lookup time.
• Let's put the resolver at the flat attrs-dict layer, not on Span. The live streaming path doesn't build Span objects, it calls the extract_*_from_attrs functions directly.
• Keep scope to the extractor for this one. Capturing schema_url + the alias resolver + surfacing the version on the conversion metadata is enough for now. Let's leave the run/storage model and any migration out of the initial PR, that's a separate thing if we ever want it.
• Note that detection. schema_url is the generic OTel schema URL, not a GenAI-specific one, so any of the scopes can have it. I wouldn't use it as a direct "this is GenAI" signal, the existing gen_ai.* check already catches normalized traces.