Skip to content

Commit df862a0

Browse files
Add 0084 orphan fallback (both observers) (#241)
* Add 0084 orphan fallback (both observers) Proposal 0084 (nested-fan-out span lineage), second of the Wave 3 PRs; completes the observer-side work started in #240. Generalize the observability orphan fallback -- a provider or framework call with no open calling-node span (a guardrail or middleware side call) -- in both the OTel and Langfuse observers to resolve the nearest enclosing wrapper on the calling node's lineage, chain-routed to the correct inner fan-out instance. A shared resolver backs both the node parent and the orphan fallback in each observer, and now also backs FailureIsolatedEvent parenting so the marker resolves consistently across observers. The Langfuse node-observation key gains the enclosing lineage chains (mirroring the OTel node-span key from #240) so nested exact-match works there too. Un-defer observability fixtures 133 (OTel orphan) and 134 (Langfuse, exact-match + orphan). The orphan-call primitive calls_llm_from_wrapper is modeled on NodeSpec so 133 and 134 both parse and run. Known limitation: a wrapper call issued before the wrapped node body (pre-phase) inside a nested fan-out resolves to the outer instance in Langfuse, which creates instance observations lazily, while OTel resolves to the inner instance -- a cross-observer parity gap the fixtures do not cover. The conformance.toml 0084 note documents it; the Langfuse eager-creation fix and the spec question are tracked as follow-ups. * Guard 0084 orphan resolution for parallel branches Add unit-test guards pinning the branch-side of the §5.5 orphan fallback in both observers -- an orphan provider call inside a parallel branch parents under the per-branch dispatch span / observation (the nearest enclosing wrapper), the branch analog of the fan-out orphan tests. The 0084 fixtures cover fan-out nesting only; a spec fixture for the branch case is flagged for the batched review (release-v0.17.0 coord). * Comment the 134 case-2 rendezvous timeout fallback (CodeQL)
1 parent 348c9d5 commit df862a0

10 files changed

Lines changed: 1270 additions & 202 deletions

File tree

conformance.toml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -836,9 +836,9 @@ note = "Advisory, observability-only per-prompt token budget. PR A (completion p
836836
# (graph-engine §6 fan_out_index_chain / branch_name_chain +
837837
# observability §4 / §5.5 lineage-resolved parent).
838838
[proposals."0084"]
839-
status = "partial"
839+
status = "implemented"
840840
since = "0.17.0"
841-
note = "The event-surface chains + the nested-LLM exact-match shipped. graph-engine §6: fan_out_index_chain / branch_name_chain (mirroring NodeEvent) now ride the provider/tool events (LlmCompletionEvent / LlmFailedEvent / LlmRetryAttemptEvent / EmbeddingEvent / EmbeddingFailedEvent / RerankEvent / RerankFailedEvent / ToolCallEvent / ToolCallFailedEvent) and the framework FailureIsolatedEvent, populated from the enclosing-lineage ContextVars at call time. observability §4.3 / §5.5: the OTel node-span key carries the scalars AND the enclosing chains, so an inner node under two concurrent outer fan-out instances no longer collides (no dropped / mis-closed spans), and the LLM-parent exact-match (§5.5 Lineage-resolved parent, fallback 1) resolves under the lineage-disambiguated calling-node span rather than the innermost scalar. observability fixture 132 (nested keying + nested-LLM exact-match, OTel) is un-deferred and driven. partial because the orphan fallback (a wrapper / middleware-issued provider call with no open calling-node span, §5.5) is not yet generalized to nearest-open-ancestor-at-any-depth and the Langfuse resolver is unchanged: observability fixtures 133 (OTel orphan) + 134 (Langfuse) and the graph-engine event-chain fixture 039 tightening defer with the orphan-call conformance-adapter primitive (a later PR)."
841+
note = "Nested-fan-out span lineage. graph-engine §6: fan_out_index_chain / branch_name_chain (mirroring NodeEvent) ride the provider/tool events (LlmCompletionEvent / LlmFailedEvent / LlmRetryAttemptEvent / EmbeddingEvent / EmbeddingFailedEvent / RerankEvent / RerankFailedEvent / ToolCallEvent / ToolCallFailedEvent) and the framework FailureIsolatedEvent, populated from the enclosing-lineage ContextVars at call time. observability §4.3 / §5.5: the OTel node-span key AND the Langfuse node-observation key now carry the scalars plus the enclosing chains, so an inner node under two concurrent outer fan-out instances no longer collides (no dropped / mis-closed spans / observations), while a callable parallel-branch stays distinct from its parallel-branches node. The LLM-parent exact-match (§5.5 Lineage-resolved parent, fallback 1) resolves under the lineage-disambiguated calling-node span / observation rather than the innermost scalar. The §5.5 orphan fallback (a wrapper / middleware-issued provider call with no open calling-node span) is generalized in BOTH observers to the nearest enclosing wrapper on the calling node's lineage via a shared resolver (_resolve_enclosing_wrapper_context on OTel, _resolve_enclosing_wrapper_observation_id on Langfuse), which also now backs FailureIsolatedEvent parenting so the marker resolves consistently across observers. observability fixtures 132 / 133 / 134 all run; the orphan-call primitive calls_llm_from_wrapper is modeled as NodeSpec.calls_llm_from_wrapper so 133 / 134 also parse + round-trip. KNOWN LIMITATION (cross-observer parity, §8.9): the harness realizes the orphan call as fan-out INSTANCE middleware in the POST phase, because a per-node PRE-phase wrapper call does not orphan in the python engine (prepare_sync opens the node span before node middleware). Under the fixtures' declared PRE phase, a Langfuse orphan resolves to the OUTER fan-out instance (Langfuse creates instance observations lazily, so the inner one does not exist when the pre-phase event drains) while OTel resolves to the INNER -- a real pre-phase parity gap the fixtures do not cover. Parity holds for the post-phase realization the fixtures drive (asserted by a live OTel-vs-Langfuse parent comparison in 134). The pre-phase gap is tracked (Langfuse eager instance-observation creation) and flagged to spec."
842842

843843
# Spec v0.80.0 (proposal 0085). Nested-fan-out checkpoint resume
844844
# lineage (pipeline-utilities §10.11 enclosing_fan_out_lineage).

src/openarmature/graph/events.py

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1142,12 +1142,13 @@ class FailureIsolatedEvent:
11421142
post_state: Mapping[str, Any]
11431143
caught_exception: CaughtException
11441144
# Proposal 0084 (spec v0.81.0): enclosing fan-out / branch lineage chains
1145-
# parallel to ``namespace`` (see LlmCompletionEvent). Carried for surface
1146-
# consistency with the provider events. The marker's calling-node span is
1147-
# already closed by delivery time (its completed event precedes this one on
1148-
# the serial queue), so it parents under the invocation span, not via the
1149-
# lineage exact-match -- the chains only disambiguate in the rare case that
1150-
# calling-node span is still open.
1145+
# parallel to ``namespace`` (see LlmCompletionEvent). The marker's
1146+
# calling-node span/observation is already closed by delivery time (its
1147+
# completed event precedes this one on the serial queue), so both observers
1148+
# take the §5.5 orphan fallback and parent the marker under the nearest
1149+
# enclosing wrapper on this lineage (the fan-out instance / branch / subgraph
1150+
# span, else the invocation span / Trace). The chains disambiguate that
1151+
# enclosing wrapper across concurrent sibling instances.
11511152
fan_out_index_chain: tuple[int | None, ...] = ()
11521153
branch_name_chain: tuple[str | None, ...] = ()
11531154

0 commit comments

Comments
 (0)