Skip to content

Emit implementation attribution on every invocation#132

Merged
chris-colinsky merged 2 commits into
mainfrom
feature/0052-implementation-attribution-attributes
Jun 6, 2026
Merged

Emit implementation attribution on every invocation#132
chris-colinsky merged 2 commits into
mainfrom
feature/0052-implementation-attribution-attributes

Conversation

@chris-colinsky

Copy link
Copy Markdown
Member

Summary

PR 3 of the v0.12.0 cycle. Last code PR before the release. Implements proposal 0052 (spec observability §5.1 + §8.4.1, spec v0.44.0).

Every OTel invocation span now carries openarmature.implementation.name ("openarmature-python") and openarmature.implementation.version (the package's __version__) alongside the existing openarmature.graph.spec_version. The Langfuse observer mirrors as trace.metadata.implementation_name / trace.metadata.implementation_version rows on every Trace. The values answer the first triage question operators ask in any observability backend: "which library, at which version, produced this trace" — without the deployment-manifest lookup.

The __implementation_name__ = "openarmature-python" constant joins __version__ and __spec_version__ at the package root. No symmetric __implementation_version__ — the value sources from __version__ directly to avoid the maintenance trap of two constants having to stay in lockstep across releases.

Coord context: discuss-trace-implementation-attribution thread; spec accepted direction in 02-spec-accept-direction.md (naming, canonical values, always-emit invariant, §3.4 reservation, OTel + Langfuse coverage).

Notable pieces

  • OTel observer (src/openarmature/observability/otel/observer.py):
    • Two helper functions _read_implementation_name() / _read_implementation_version() mirroring the existing _read_spec_version() lazy-import pattern.
    • Two new dataclass fields implementation_name / implementation_version configurable for test parameterization, defaulting to the package identity.
    • Four-line addition to _open_invocation_span (line 1266) emitting both attributes on the invocation span. Inner-node spans don't carry them — per §5.1 they are invocation-span-only, not the cross-cutting §5.6 family.
  • Langfuse observer (src/openarmature/observability/langfuse/observer.py):
    • Same helper + field pattern.
    • Both trace-open paths emit the rows: _open_trace (legacy NodeEvent path) AND _lazy_open_trace_for_boundary_event (proposal 0043 invocation-boundary path).
  • Reserved-key extension (src/openarmature/observability/metadata.py): _RESERVED_KEY_NAMES grows 24 → 26. Caller passing invocation_metadata={"implementation_name": "spoof"} gets ValueError at the invoke() boundary — same enforcement path as the other 24 reserved names.
  • Always-emit invariant: neither disable_state_payload, disable_llm_payload, disable_llm_spans, nor disable_genai_semconv gates the attributes. They describe runtime identity, not runtime data.

Behavior pins

Twelve new unit tests:

  • 6 reserved-key tests in tests/unit/test_observability_metadata.py: validate / set_invocation_metadata / invoke() boundary rejection for both implementation_name and implementation_version.
  • 3 OTel tests in tests/unit/test_observability_otel.py:
    • test_invocation_span_carries_implementation_attribution_attributes — attributes present on invocation span, absent on inner spans, name matches "openarmature-python".
    • test_invocation_span_attribution_emits_under_disable_llm_payload — always-emit invariant under three privacy knobs (disable_llm_payload, disable_genai_semconv, disable_llm_spans).
    • test_invocation_span_attribution_emits_on_every_invocation — multi-invocation reuse: 3 invocations on the same observer, every invocation span carries the attributes.
  • 3 Langfuse tests in tests/unit/test_observability_langfuse.py:
    • test_trace_metadata_carries_implementation_attribution_rows — both rows present on the Trace, name matches the canonical value.
    • test_implementation_attribution_rows_emit_with_disable_state_payload_enabled — always-emit under the §8.4.1 state-payload privacy knob.
    • test_implementation_attribution_rows_emit_on_every_trace — multi-invocation reuse: 3 invocations on the same observer, every Trace carries the rows.

Spec-fixture inconsistency surfaced during this PR

Spec fixture observability/058-implementation-attribution-otel.yaml case 2 expects two openarmature.invocation spans on a detached-subgraph composition (one per trace). Spec fixture observability/008-otel-detached-trace-mode.yaml establishes that detached subgraphs root in their subgraph-dispatch span, NOT in a separate invocation span. The two fixtures appear inconsistent. The python implementation honors fixture 008's model (one invocation span per detached composition), so trying to pin "two invocation spans" via a unit test would fail against the existing implementation. The 0052 contract itself — "every invocation span carries the attribution attributes" — is honored regardless of which invocation-span model wins. Flagged in the conformance.toml 0052 comment block for spec-side resolution in a separate coord thread. Not blocking this PR.

Conformance + docs

  • conformance.toml: [proposals."0052"] flips not-yetimplemented/since="0.12.0". Comment block enumerates the 12 unit-test pins and documents the fixture 058 vs 008 inconsistency.
  • tests/conformance/test_fixture_parsing.py: observability/059 deferral rationale updates from "lands in PR 3 of v0.12.0" to "fixture-shape models pending; contract pinned by unit tests". 058 parses cleanly against existing harness shapes; runtime exec is gated by _SUPPORTED_FIXTURES in test_observability.py.
  • CHANGELOG.md: ### Added entry under [Unreleased] positioned before the PR 2/2b entries.
  • src/openarmature/AGENTS.md: regenerated (idempotent — same content).

Out of scope

  • Runtime attribution (Python version, OS, host) — per spec, separate axis.
  • Compiled-graph identity — per spec, separate axis.
  • Fixture 058/059 runtime exec activation — Path A defer pending the upcoming conformance-adapter capability ratifying the harness_parameterized directive.
  • Fixture 058 vs 008 invocation-span-model inconsistency — coord-thread + spec resolution in a separate PR.

Test plan

  • uv run pytest tests/ — 1123 passed (was 1111 pre-PR), 307 skipped, 0 failures
  • uv run pytest tests/unit/test_observability_metadata.py tests/unit/test_observability_otel.py tests/unit/test_observability_langfuse.py — 131 passed (was 119; +12 new)
  • uv run python scripts/check_conformance_manifest.py — 51/51 entries consistent
  • uv run ruff check . + uv run ruff format --check . — clean
  • uv run pyright src/ tests/ — 0 errors
  • uv run mkdocs build --strict — clean

Implements proposal 0052 (spec observability 5.1 + 8.4.1,
v0.44.0). Two new attributes -- openarmature.implementation.name
and openarmature.implementation.version -- emit on every OTel
invocation span alongside the existing openarmature.graph.spec_version.
The Langfuse observer mirrors the rows as
trace.metadata.implementation_name and trace.metadata.implementation_version
on every Trace. The values let observability-backend operators
answer "which library, at which version, produced this trace"
without a separate deployment-manifest lookup.

The name value is the canonical PyPI package name
("openarmature-python"), pinned as a new __implementation_name__
constant at the package root next to __version__ and
__spec_version__. The version value is __version__ itself -- no
separate __implementation_version__ constant to avoid the
maintenance trap of two values that have to stay in lockstep.

Both observers expose configurable implementation_name and
implementation_version dataclass fields for test parameterization,
defaulting to the package identity via lazy-import helpers that
mirror the existing _read_spec_version pattern. The OTel invocation
span emits the attributes once per invocation (per 5.1, not the
cross-cutting 5.6 family -- inner-node spans do NOT carry them).
The Langfuse rows emit on both trace-open paths: the proposal 0043
boundary-event lazy path and the legacy NodeEvent path.

Always-emit invariant pinned on both observers: neither
disable_state_payload, disable_llm_payload, disable_llm_spans, nor
disable_genai_semconv gates the attributes, since they describe
runtime identity rather than runtime data.

The 3.4 reserved-key set extends from 24 to 26 names:
implementation_name and implementation_version reject
caller-supplied collision at the invoke() boundary, so a caller
passing invocation_metadata={"implementation_name": "spoof"}
gets ValueError rather than silently clobbering the
implementation-emitted value.

Twelve new unit tests pin the contract: 6 reserved-key (validate /
set / invoke boundary for both names) + 3 OTel (invocation-span
presence + inner-span absence, always-emit under disable_llm_payload,
multi-invocation reuse) + 3 Langfuse (trace metadata presence,
always-emit under disable_state_payload, multi-invocation reuse).
Conformance fixtures 058 (OTel) + 059 (Langfuse) stay deferred
from the cross-capability parser pending the upcoming
conformance-adapter capability spec.
Copilot AI review requested due to automatic review settings June 6, 2026 00:07

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements proposal 0052 by emitting implementation attribution (name + version) on every invocation span/trace, enabling operators to identify which OpenArmature library/version produced observability data without external lookups.

Changes:

  • Add openarmature.implementation.name / openarmature.implementation.version to OTel invocation spans and mirror as trace.metadata.implementation_name / trace.metadata.implementation_version in Langfuse.
  • Reserve implementation_name / implementation_version in invocation metadata to prevent caller clobbering.
  • Mark proposal 0052 as implemented in conformance.toml, add/adjust tests and docs/changelog notes.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/openarmature/observability/otel/observer.py Adds lazy-read helpers, observer fields, and emits implementation attribution attributes on invocation spans.
src/openarmature/observability/langfuse/observer.py Adds lazy-read helpers/fields and emits implementation attribution rows on trace open (both paths).
src/openarmature/observability/metadata.py Reserves implementation_name / implementation_version to prevent metadata collisions.
src/openarmature/__init__.py Introduces __implementation_name__ constant at package root.
tests/unit/test_observability_otel.py Adds unit tests asserting invocation-only emission and always-emit behavior across invocations/privacy knobs.
tests/unit/test_observability_langfuse.py Adds unit tests asserting trace metadata rows and always-emit behavior across invocations/privacy knobs.
tests/unit/test_observability_metadata.py Adds unit tests for reserved-key rejection for the new reserved names.
tests/conformance/test_fixture_parsing.py Updates deferral rationale/comments for proposal 0052 fixture parsing.
conformance.toml Flips proposal 0052 to implemented and documents behavior/test pins (plus notes on fixture inconsistency).
CHANGELOG.md Adds an “Implementation attribution attributes” entry under Unreleased.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread conformance.toml Outdated
Comment thread src/openarmature/observability/otel/observer.py
Comment thread src/openarmature/observability/langfuse/observer.py
Three documentation consistency fixes from PR review.

OTelObserver and LangfuseObserver gained public
implementation_name / implementation_version constructor
parameters in the prior commit, but each class docstring's
"Constructor knobs" list was not updated. Adding entries to
both, slotted after spec_version (OTel) and after the trace
input/output hooks (Langfuse) so the always-emit knobs group
with the other invocation-span / trace-metadata sourcing
parameters.

The conformance.toml proposal 0052 comment block claimed both
observability/058 and /059 were deferred from the cross-
capability parser, but later in the same block stated 058
parses cleanly. Tightening the opening sentence so 058 is
correctly described as parseable-but-runtime-gated and 059 as
parser-deferred.
@chris-colinsky chris-colinsky merged commit a283e62 into main Jun 6, 2026
6 checks passed
@chris-colinsky chris-colinsky deleted the feature/0052-implementation-attribution-attributes branch June 6, 2026 00:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants