Tighten v0.84.0 deferral accuracy + dedup 008

From a PR #197 review: several deferral reasons and conformance.toml
statuses overstated non-implementation for partially-shipped work.

- conformance.toml: 0081 not-yet -> textual-only (the <any-string>
  non-empty matcher is enforced by the adapter; precedent 0055/0071);
  0085 not-yet -> partial (the save-side enclosing_fan_out_lineage
  keying shipped in #194; only the resume consume-side is unshipped),
  and drop the false "fixture 076 defers with it" claim (076 is
  number-gated, not deferred); drop the stale "a new case on 008"
  from the 0084 comment (that case is 0061's and is now wired).
- Reword the 132 / 023-027 / 119 / 022-023 deferral reasons to state
  what actually ships (the 0084 lineage keying, the 0059 embed wire,
  0075 callable branches, the structured_output_invalid base mapping)
  versus what is genuinely deferred.
- Extract _assert_detached_raise_both_spans from the two near-identical
  008 raise-case blocks, and add the parent openarmature.invocation
  ERROR assertion the fixtures' span_trees expect.

Author: chris-colinsky

conformance.toml

@@ -801,9 +801,14 @@ status = "not-yet"
 
 # Spec v0.76.0 (proposal 0081).  Conformance-adapter value-matcher
 # vocabulary (§5.10) -- ratifies the fixture matcher tokens already in
-# use.  Not-yet (no fixture changes); awaits a conformance-pin batch.
+# use.  Descriptive (no fixture changes); the one tightening with teeth
+# (<any-string> = non-empty) is already enforced by the adapter
+# (tests/conformance/test_observability.py _value_matches rejects the
+# empty string). No module-level change required; matches the
+# conformance-adapter textual-only precedent (0055 / 0071).
 [proposals."0081"]
-status = "not-yet"
+status = "textual-only"
+since = "0.16.0"
 
 # Spec v0.77.0 (proposal 0082).  Structured-output failure diagnostics
 # (graph-engine §6 -- LlmFailedEvent response-side surface for
@@ -822,16 +827,16 @@ status = "not-yet"
 # Spec v0.81.0 (proposal 0084).  Nested-fan-out span lineage chain
 # (graph-engine §6 fan_out_index_chain / branch_name_chain +
 # observability §4 / §5.5 lineage-resolved parent).  Not-yet; graph-
-# engine fixture 039 and observability fixtures 132-134 (+ a new case
-# on 008) defer with it.
+# engine fixture 039 and observability fixtures 132-134 defer with it.
 [proposals."0084"]
 status = "not-yet"
 
 # Spec v0.80.0 (proposal 0085).  Nested-fan-out checkpoint resume
-# lineage (pipeline-utilities §10.11 enclosing_fan_out_lineage).  Not-
-# yet; pipeline-utilities fixture 076 defers with it.
+# lineage (pipeline-utilities §10.11 enclosing_fan_out_lineage).
 [proposals."0085"]
-status = "not-yet"
+status = "partial"
+since = "0.16.0"
+note = "The SAVE-side enclosing_fan_out_lineage keying (pipeline-utilities §10.11) shipped in #194: a fan-out instance's checkpoint tracking key carries the enclosing fan-out instance lineage in the in-memory dict and through the checkpoint projection / lookup / cleanup / restore, so concurrent outer instances no longer collide. partial because the RESUME consume-side is not yet shipped: a fan-out nested inside an outer instance re-runs rather than skipping on resume, since the saved record format carries no lineage (tracked as a follow-up). pipeline-utilities fixture 076 is not collected by the test_pipeline_utilities.py _LAST_DRIVEN_FIXTURE number gate (it is not deferred); the resume consume-side plus its fixture wiring land in a later PR."
 
 # Spec v0.79.0 (proposal 0086).  Service-wide default cache_ttl_seconds
 # on PromptManager (prompt-management §6).  Not-yet; prompt-management

tests/conformance/test_llm_provider.py

@@ -115,9 +115,13 @@
     # ----- v0.16.0 spec-pin bump (v0.70.1 -> v0.84.0) -------------------
     # Proposal 0082 (structured-output failure diagnostics, spec v0.77.0)
     # extended fixtures 022/023 to additionally assert the now-required
-    # finish_reason + usage on the structured_output_invalid error; python
-    # does not yet carry those response-side fields on the error, so the
-    # extended fixtures defer until a later v0.16.0 PR lands 0082.
+    # finish_reason + usage on the structured_output_invalid error. The
+    # base structured_output_invalid mapping still has coverage in
+    # tests/unit/test_structured_output.py::
+    # test_pydantic_validation_failure_wraps_in_structured_output_invalid;
+    # only 0082's additive finish_reason / usage-on-error response-side
+    # fields are unimplemented, which is why the conformance fixtures defer
+    # until a later v0.16.0 PR lands 0082.
     "022-structured-output-parse-failure": (
         "Proposal 0082 finish_reason/usage on structured_output_invalid; not implemented"
     ),

tests/conformance/test_observability.py

@@ -316,7 +316,8 @@ def _reset_otel_global_tracer_provider(restore_to: object) -> None:
     # shape the cross-capability parser doesn't model (cf. 110) and has no
     # dedicated runner here yet; defer until the harness wires it.
     "119-otel-callable-branch-attempt-index-under-node-retry": (
-        "proposal 0075 callable-branch attempt_index coverage round-out; harness runner not yet wired"
+        "0075 shipped v0.15.0; runner for this callable-branch-attempt-index-under-retry "
+        "case not yet wired -- harness gap, not unimplemented"
     ),
     # Proposal 0082 (structured-output failure diagnostics, spec v0.77.0).
     # The LlmFailedEvent response-side surface (output_content / finish_reason
@@ -347,15 +348,24 @@ def _reset_otel_global_tracer_provider(restore_to: object) -> None:
             "131-token-budget-on-structured-output-failure",
         )
     },
-    # Proposal 0084 (nested-fan-out span lineage, spec v0.81.0). The
-    # fan_out_index_chain / branch_name_chain event surface + the lineage-
-    # keyed OTel parent resolution are unimplemented until a later v0.16.0
-    # PR; the no-dropped-spans keying, orphan fallback, and Langfuse parent-
-    # resolution fixtures defer with it.
+    # Proposal 0084 (nested-fan-out span lineage, spec v0.81.0).
+    # Fixture 132's nested-lineage dispatch KEYING shipped in #194 (the
+    # OTel + Langfuse observers key dispatches by the full enclosing
+    # fan-out / branch lineage, so no spans drop). It stays deferred for
+    # 0084's remaining surface: the fan_out_index_chain / branch_name_chain
+    # arrive on the provider events with 0084 (see graph/events.py), and the
+    # nested-orphan LLM parent resolution is top-level-instance-only as of
+    # #195 -- the at-any-depth generalization rides 0084.
+    "132-otel-nested-fan-out-span-keying-and-llm-exact-match": (
+        "nested-lineage span keying shipped (#194); deferred for 0084's event-chain "
+        "surface (fan_out_index_chain / branch_name_chain on provider events) and the "
+        "nested-orphan LLM parent (top-level only as of #195)"
+    ),
+    # Proposal 0084: 133 (orphan LLM fallback) + 134 (Langfuse parent
+    # resolution) defer with 0084's lineage-resolved parent surface.
     **{
         fixture_id: "nested-fan-out span lineage (proposal 0084) not-yet implemented"
         for fixture_id in (
-            "132-otel-nested-fan-out-span-keying-and-llm-exact-match",
             "133-otel-nested-fan-out-orphan-llm-fallback",
             "134-langfuse-nested-fan-out-parent-resolution",
         )
@@ -2539,6 +2549,54 @@ def _has_exception_event(span: Any) -> bool:
     return any(getattr(e, "name", None) == "exception" for e in events)
 
 
+def _assert_detached_raise_both_spans(
+    spans: Sequence[ReadableSpan],
+    *,
+    parent_span_name: str,
+    expected_link_count: int,
+) -> None:
+    # Proposal 0061 §4.2: a raising detached unit surfaces ERROR on BOTH the
+    # parent's dispatch span (the one carrying the Link, named
+    # ``parent_span_name``) and the per-unit detached invocation span (its own
+    # trace) -- distinct traces, shared invocation_id, each with the §4 category
+    # + an OTel exception event. The parent's own openarmature.invocation span
+    # also inherits ERROR (§4.4): the fixtures' expected span_trees show the
+    # parent invocation ERROR.
+    parent_dispatch = next((s for s in spans if s.name == parent_span_name and s.links), None)
+    assert parent_dispatch is not None, f"expected a parent {parent_span_name!r} span carrying a Link"
+    assert parent_dispatch.status.status_code.name == "ERROR", (
+        f"parent {parent_span_name!r} span MUST carry ERROR for a raising detached unit"
+    )
+    assert _has_exception_event(parent_dispatch), (
+        f"parent {parent_span_name!r} span MUST record the exception"
+    )
+    assert dict(parent_dispatch.attributes or {}).get("openarmature.error.category") == "node_exception"
+    assert len(parent_dispatch.links) == expected_link_count, (
+        f"parent {parent_span_name!r} span MUST carry exactly {expected_link_count} Link(s); "
+        f"got {len(parent_dispatch.links)}"
+    )
+    parent_trace_id = cast("Any", parent_dispatch.context).trace_id
+    detached_trace_id = parent_dispatch.links[0].context.trace_id
+    assert detached_trace_id != parent_trace_id, "detached + parent traces MUST be distinct"
+    inv_spans = [s for s in spans if s.name == "openarmature.invocation"]
+    detached_inv = next((s for s in inv_spans if cast("Any", s.context).trace_id == detached_trace_id), None)
+    parent_inv = next((s for s in inv_spans if cast("Any", s.context).trace_id == parent_trace_id), None)
+    assert detached_inv is not None, "detached trace MUST root in an openarmature.invocation span"
+    assert parent_inv is not None
+    assert parent_inv.status.status_code.name == "ERROR", (
+        "parent openarmature.invocation span MUST inherit ERROR for a raising detached unit (§4.4)"
+    )
+    assert detached_inv.status.status_code.name == "ERROR", (
+        "detached invocation span MUST carry the detached unit's ERROR status (§4.2)"
+    )
+    assert _has_exception_event(detached_inv), "detached invocation span MUST record the exception"
+    assert dict(detached_inv.attributes or {}).get("openarmature.error.category") == "node_exception"
+    parent_iid = _invocation_id_of(parent_inv)
+    assert parent_iid is not None and _invocation_id_of(detached_inv) == parent_iid, (
+        "detached invocation span MUST share the parent's invocation_id"
+    )
+
+
 async def _run_fixture_008_case(case: Mapping[str, Any]) -> None:
     case_name = case["name"]
     expect_raise = case_name in (
@@ -2644,79 +2702,17 @@ async def _run_fixture_008_case(case: Mapping[str, Any]) -> None:
         return
 
     if case_name == "detached_subgraph_raises_error_status_on_both_spans":
-        # Proposal 0061 §4.2: a raising detached subgraph surfaces ERROR
-        # on BOTH the parent's dispatch span and the detached invocation
-        # span — distinct traces, shared invocation_id, each with the §4
-        # category + an OTel exception event.
-        dispatch_spans = [s for s in spans if s.name == "dispatch"]
-        parent_dispatch = next((s for s in dispatch_spans if s.links), None)
-        assert parent_dispatch is not None, "expected a parent 'dispatch' span with a Link"
-        assert parent_dispatch.status.status_code.name == "ERROR", (
-            "parent dispatch span MUST carry ERROR for a raising detached subgraph"
-        )
-        assert _has_exception_event(parent_dispatch), "parent dispatch span MUST record the exception"
-        assert dict(parent_dispatch.attributes or {}).get("openarmature.error.category") == "node_exception"
-        parent_trace_id = cast("Any", parent_dispatch.context).trace_id
-        detached_trace_id = parent_dispatch.links[0].context.trace_id
-        assert detached_trace_id != parent_trace_id, "detached + parent traces MUST be distinct"
-        inv_spans = [s for s in spans if s.name == "openarmature.invocation"]
-        detached_inv = next(
-            (s for s in inv_spans if cast("Any", s.context).trace_id == detached_trace_id), None
-        )
-        parent_inv = next((s for s in inv_spans if cast("Any", s.context).trace_id == parent_trace_id), None)
-        assert detached_inv is not None, "detached trace MUST root in an openarmature.invocation span"
-        assert parent_inv is not None
-        assert detached_inv.status.status_code.name == "ERROR", (
-            "detached invocation span MUST carry the detached unit's ERROR status (§4.2)"
-        )
-        assert _has_exception_event(detached_inv), "detached invocation span MUST record the exception"
-        assert dict(detached_inv.attributes or {}).get("openarmature.error.category") == "node_exception"
-        parent_iid = _invocation_id_of(parent_inv)
-        assert parent_iid is not None and _invocation_id_of(detached_inv) == parent_iid, (
-            "detached invocation span MUST share the parent's invocation_id"
-        )
+        # The parent's subgraph-dispatch span (named "dispatch") carries the
+        # single Link to the one detached subgraph trace.
+        _assert_detached_raise_both_spans(spans, parent_span_name="dispatch", expected_link_count=1)
         return
 
     if case_name == "detached_fan_out_instance_raises_error_status_on_both_spans":
-        # Proposal 0061 §4.2, fan-out-instance variant: a raising detached
-        # fan-out instance surfaces ERROR on BOTH the parent's fan-out node
-        # span (parent trace, carrying the Link) and the per-instance detached
-        # invocation span (its own trace), each with the §4 category + an OTel
-        # exception event, sharing the parent invocation_id. The single-element
-        # fan-out (items [1]) means exactly one instance runs and raises.
-        fan_out_node_spans = [s for s in spans if s.name == "per_document_scoring"]
-        parent_fan_out = next((s for s in fan_out_node_spans if s.links), None)
-        assert parent_fan_out is not None, "expected a fan-out node span with a Link in the parent trace"
-        assert parent_fan_out.status.status_code.name == "ERROR", (
-            "parent fan-out node span MUST carry ERROR for a raising detached instance"
-        )
-        assert _has_exception_event(parent_fan_out), "parent fan-out node span MUST record the exception"
-        assert dict(parent_fan_out.attributes or {}).get("openarmature.error.category") == "node_exception"
-        assert len(parent_fan_out.links) == 1, (
-            f"fan-out node span MUST carry exactly one Link; got {len(parent_fan_out.links)}"
-        )
-        parent_trace_id = cast("Any", parent_fan_out.context).trace_id
-        detached_trace_id = parent_fan_out.links[0].context.trace_id
-        assert detached_trace_id != parent_trace_id, "detached instance + parent traces MUST be distinct"
-        inv_spans = [s for s in spans if s.name == "openarmature.invocation"]
-        detached_inv = next(
-            (s for s in inv_spans if cast("Any", s.context).trace_id == detached_trace_id), None
-        )
-        parent_inv = next((s for s in inv_spans if cast("Any", s.context).trace_id == parent_trace_id), None)
-        assert detached_inv is not None, (
-            "detached instance trace MUST root in an openarmature.invocation span"
-        )
-        assert parent_inv is not None
-        assert detached_inv.status.status_code.name == "ERROR", (
-            "detached instance invocation span MUST carry the instance's ERROR status (§4.2)"
-        )
-        assert _has_exception_event(detached_inv), (
-            "detached instance invocation span MUST record the exception"
-        )
-        assert dict(detached_inv.attributes or {}).get("openarmature.error.category") == "node_exception"
-        parent_iid = _invocation_id_of(parent_inv)
-        assert parent_iid is not None and _invocation_id_of(detached_inv) == parent_iid, (
-            "detached instance invocation span MUST share the parent's invocation_id"
+        # Fan-out-instance variant: the parent's fan-out node span
+        # ("per_document_scoring") carries the Link. The single-element fan-out
+        # (items [1]) means exactly one instance runs and raises -> one Link.
+        _assert_detached_raise_both_spans(
+            spans, parent_span_name="per_document_scoring", expected_link_count=1
         )
         return
 

tests/conformance/test_retrieval_provider.py

@@ -65,7 +65,12 @@
         if 18 <= int(p.stem[:3]) <= 22
     },
     **{
-        p.stem: "OpenAI-compatible embeddings wire mapping (proposal 0079) not implemented"
+        p.stem: (
+            "OpenAI-compatible embed wire ships via the bundled OpenAIEmbeddingProvider "
+            "(proposal 0059); deferred because the harness lacks a wire-capture primitive "
+            "(expected_wire_request / url / headers) and 0079's dimensions / input_type "
+            "request knobs are unimplemented"
+        )
         for p in CONFORMANCE_DIR.glob("[0-9][0-9][0-9]-*.yaml")
         if 23 <= int(p.stem[:3]) <= 27
     },