Add Langfuse observer (proposal 0031) (#80)

* Add Langfuse observer (proposal 0031)

New openarmature.observability.langfuse subpackage maps the observer
event stream onto Langfuse's native Trace + Observation data model:
invocation -> Trace, node/subgraph/fan-out -> Span observation, LLM
provider call -> Generation observation.

The LangfuseObserver consumes the event stream and emits through a
narrow LangfuseClient Protocol with four methods (trace, span,
generation, update_trace). Two concrete clients:
- InMemoryLangfuseClient bundled for tests and the conformance
  harness; captures every Trace + Observation as plain dataclass
  records inspectable by assertions.
- A real langfuse.Langfuse() SDK instance is Protocol-compatible
  and drops in unchanged for production use. SDK versions whose
  shape diverges plug in via a small adapter the user writes; the
  shape is documented in examples/10-langfuse-observability.

Trace id equals the framework-minted invocation_id verbatim so
cross-system lookup by invocation_id finds the Langfuse Trace
directly. correlation_id surfaces on both trace.metadata and every
observation.metadata for cross-backend join. Trace name sources
from the entry-node name; the caller-supplied invocation-label
path lands in proposal 0034 (PR 4).

Generation rendering follows the §8.7 contract: input/output/
request_extras appear only when disable_llm_payload=False; the
§5.5.5 truncation marker is preserved verbatim as a raw string
when the serialized payload exceeds payload_byte_cap.

Prompt linkage follows §8.4.4 case discrimination: reads
Prompt.observability_entities['langfuse_prompt'] (the field added
in proposal 0033) to establish a native Prompt-entity link when
the prompt's source exposes one. Backends without that reference
(filesystem, in-memory) produce metadata-only linkage.

Three conformance fixtures pass: 022 basic trace, 023 Generation
rendering plus truncation case, 024 prompt linkage both cases.
Seven new unit tests cover payload-cap validation, in-memory
recorder field handling, and observation parent walking.

Example 10-langfuse-observability is a runnable moon-themed demo:
a lunar mission Q&A pipeline with a mock Langfuse-source prompt
backend, the LangfuseObserver wired to the in-memory recorder,
and a pretty-printer for the captured Trace tree at the end. Real
SDK swap is a one-line constructor change.

Third of 6 PRs in the v0.10.0 batch.

* Document deferred behavior in Langfuse observer

Four doc-only additions from PR review:

- _resolve_parent_observation_id now explicitly notes the
  longest-prefix-first outer-loop walk and calls out the first-match
  approximation when multiple open observations share a namespace
  prefix. The proper subgraph_observations /
  fan_out_instance_observations / detached_roots maps land in a
  follow-on PR; the current resolver covers the linear-graph and
  basic-LLM cases the v0.23.0 conformance fixtures exercise.

- _maybe_truncate_for_input and _maybe_truncate_for_extras docstrings
  document the intentional list|str and dict|str union return types.
  When the serialized payload exceeds payload_byte_cap the helpers
  return the marker-bearing string verbatim per spec §8.7; the
  unparseable JSON IS the truncation signal and the Langfuse UI
  renders the string view in that case.

- LangfuseClient.update_trace gets a comment explaining why it's
  declared in the Protocol despite the current observer not invoking
  it. The caller-supplied invocation-label path (proposal 0034, PR 4
  of the v0.10.0 batch) may need to swap the trace name after the
  first node event opens the Trace; SDK adapters implement it for
  forward compatibility with that wiring.

- Concepts and example docs now disclose that no specific langfuse
  SDK version is validated in CI for this release. A follow-on
  release pins a tested [langfuse] extras range and ships a runtime
  Protocol-conformance check; until then production wire-up is a
  "verify in your own environment" path.

Behavior unchanged. AGENTS.md regenerated to pick up the example
doc note.

Author: chris-colinsky

docs/concepts/observability.md

@@ -586,3 +586,113 @@ appear dropped. Two workarounds:
 - Use `SimpleSpanProcessor` instead of `BatchSpanProcessor` in
   tests; it exports synchronously and is unaffected by teardown
   timing.
+
+## Langfuse mapping (opt-in)
+
+A second sibling observer maps the same `NodeEvent` stream onto
+Langfuse's native Trace + Observation data model — Traces at the
+top, Span observations for graph nodes, Generation observations for
+LLM calls. Use it instead of (or alongside) the OTel observer when
+your trace UI is Langfuse and you want first-class Generation
+rendering without going through Langfuse's OTLP ingest.
+
+```python
+from openarmature.observability.langfuse import (
+    InMemoryLangfuseClient,
+    LangfuseObserver,
+)
+
+client = InMemoryLangfuseClient()  # or langfuse.Langfuse(...) in prod
+observer = LangfuseObserver(client=client)
+graph.attach_observer(observer)
+```
+
+The `client` is anything matching the `LangfuseClient` Protocol —
+the bundled `InMemoryLangfuseClient` (used by the conformance
+harness, useful for unit tests), or a real `langfuse.Langfuse()`
+instance from the [Langfuse Python SDK](https://github.com/langfuse/langfuse-python).
+The Protocol declares only the methods the observer calls, so SDK
+versions whose shape matches drop in directly. SDK versions whose
+shape diverges (renamed kwargs, return-type quirks) plug in via a
+small adapter; see
+[`examples/10-langfuse-observability`](../examples/10-langfuse-observability.md)
+for the runnable demo plus the adapter shape.
+
+!!! note "Langfuse SDK version compatibility"
+
+    No specific `langfuse` SDK version is validated in CI as of this
+    release. The Protocol mirrors the SDK's documented low-level
+    `trace` / `span` / `generation` shape, but the SDK has shifted
+    between major versions (v2 → v3 introduced API changes). A
+    follow-on release pins a tested `[langfuse]` extras range and
+    ships a runtime `isinstance` check confirming the SDK satisfies
+    the Protocol. Until then, treat production wire-up as a "verify
+    in your own environment" path: bring the langfuse version your
+    stack already uses, run a smoke trace, and write a thin adapter
+    if any kwargs don't line up.
+
+### What Langfuse sees
+
+- **Trace ID = invocation ID.** The Trace's `id` is the OA
+  `invocation_id` verbatim, so cross-system lookup by invocation_id
+  finds the Langfuse Trace directly (spec §8.4.1).
+- **Trace name.** Defaults to the entry-node name (spec §8.6
+  fallback). Caller-supplied invocation labels land in PR 4
+  (proposal 0034).
+- **Per-observation metadata.** Each Span / Generation carries
+  `namespace`, `step`, `attempt_index`, optional `fan_out_index` /
+  `branch_name`, and the `correlation_id` cross-cutting join key
+  (spec §8.5).
+- **Generation fields.** LLM calls become Generation observations
+  with `model`, `model_parameters` (the `gen_ai.request.*` request
+  parameters lifted by inclusion per §8.4.3), `usage` (input /
+  output / total tokens), and `metadata.finish_reason` /
+  `system` / `response_model` / `response_id`.
+
+### Payload + truncation
+
+`disable_llm_payload` mirrors the OTel observer's flag — defaults
+to `True` for the same privacy reason. Flip to `False` to populate
+`generation.input` / `output` / `metadata.request_extras` from the
+LLM event payload.
+
+```python
+observer = LangfuseObserver(
+    client=client,
+    disable_llm_payload=False,
+    payload_byte_cap=65536,
+)
+```
+
+When a payload exceeds `payload_byte_cap`, the observer emits the
+serialized form with the §5.5.5 truncation marker
+(`…[truncated, M bytes total]`) verbatim as a raw string instead of
+parsing back to native shape. The unparseable JSON IS the
+truncation signal in the Langfuse UI.
+
+### Prompt linkage
+
+When a Prompt's source backend exposes a Langfuse Prompt entity
+reference under `Prompt.observability_entities['langfuse_prompt']`,
+the Generation observation links to that entity natively (spec
+§8.4.4 case 1). Backends that don't surface a Langfuse reference
+(filesystem, in-memory, etc.) leave the Generation with
+`metadata.prompt` populated but no entity link (case 2).
+
+### Composition with OTel
+
+The two observers are independent §6 event consumers and can be
+attached together. They share the `correlation_id` as the
+cross-backend join key — find a slow Generation in Langfuse, search
+for its `correlation_id` in OTel logs, see the surrounding
+infrastructure activity.
+
+```python
+otel_observer = OTelObserver(span_processor=...)
+langfuse_observer = LangfuseObserver(client=langfuse_client)
+graph.attach_observer(otel_observer)
+graph.attach_observer(langfuse_observer)
+```
+
+Each observer's `disable_llm_spans` / `disable_llm_payload` flag is
+independent; one MAY emit while the other suppresses.

docs/examples/10-langfuse-observability.md

@@ -0,0 +1,168 @@
+# 10 - Langfuse observability
+
+Send LLM call observability to Langfuse natively — Trace at the top,
+Span observations for graph nodes, Generation observations with input,
+output, token usage, model parameters, and a native link back to the
+prompt entity the call rendered from.
+
+## Overview
+
+A mission-briefing assistant answers questions about Apollo and Artemis
+missions. The pipeline fetches a versioned prompt template, renders it
+with the user's question, sends the rendered messages to the model,
+and stores the response. The Langfuse observer captures the full call
+shape as the graph runs.
+
+The demo's prompt backend stubs a Langfuse-source by attaching a
+sentinel `langfuse_prompt` reference to the rendered prompt. The
+Generation observation reads that reference and links back to the
+prompt entity — exactly what you'd see in a production Langfuse
+dashboard threading "this generation came from prompt v7" without any
+manual wiring at the call site.
+
+## What it teaches
+
+- [`LangfuseObserver`](../concepts/observability.md#langfuse-mapping-opt-in)
+  attaches like any other observer; nothing in the node code knows or
+  cares about which backend is recording.
+- The `LangfuseClient` Protocol decouples the observer from the SDK.
+  The bundled `InMemoryLangfuseClient` recorder is the test/demo
+  shape; production passes a real `langfuse.Langfuse()` instance (or
+  a thin adapter — see [Reading the output](#reading-the-output)
+  below).
+- Prompt linkage through
+  [`Prompt.observability_entities`](../concepts/prompts.md#backend-keyed-observability-entity-references):
+  a prompt backend that exposes a Langfuse Prompt entity reference
+  surfaces it on every Generation that renders from that prompt.
+  Filesystem / in-memory backends without that reference work too,
+  they just produce metadata-only linkage.
+- `disable_llm_payload=False` opt-in for capturing input messages +
+  output content on Generation observations. Default-off is the
+  privacy posture; the demo deliberately flips it.
+- `correlation_id` cross-cutting metadata on the Trace and every
+  Observation — the join key if you're also running an OTel observer
+  alongside.
+
+## How to run
+
+```bash
+uv sync --group examples
+LLM_API_KEY=sk-... uv run python examples/10-langfuse-observability/main.py \
+  "what year did Apollo 11 land"
+```
+
+The first positional arg becomes the question. The demo uses an
+in-memory recorder so no Langfuse account is needed.
+
+## The graph
+
+```mermaid
+flowchart TD
+  start([start])
+  answer[answer_briefing]
+  stop([end])
+
+  start --> answer --> stop
+```
+
+A single-node graph: fetch the prompt, render with the question, call
+the LLM under `with_active_prompt(...)`, store the response. The
+single node is deliberate — the value is in the captured Trace shape,
+not the graph topology.
+
+## Reading the output
+
+After the answer prints, the script renders the captured Langfuse
+Trace + Observation tree:
+
+```
+question: what year did Apollo 11 land
+answer:   Apollo 11 landed on the Moon on July 20, 1969 ...
+prompt:   mission-briefing v7
+
+─── captured Langfuse trace ─────────────────────────────────
+Trace id=01234567-89ab-...
+      name='answer_briefing'
+      metadata={correlation_id='...', entry_node='answer_briefing', spec_version='0.26.0'}
+  [span] 'answer_briefing' level=DEFAULT
+    metadata={attempt_index=0, correlation_id='...', namespace=['answer_briefing'], step=0}
+    [generation] 'openarmature.llm.complete' level=DEFAULT
+      metadata={correlation_id='...', finish_reason='stop', prompt={...},
+                response_id='...', response_model='gpt-4o-mini-2024-...',
+                system='openai'}
+      model='gpt-4o-mini'
+      usage=input:48 output:32 total:80
+      prompt_entity_link='lf-prompt-mission-briefing-v7'
+      output='Apollo 11 landed on the Moon on July 20, 1969 ...'
+```
+
+- **Trace name = entry node name** by default. The caller-supplied
+  invocation-label path (a per-`invoke()` argument that overrides the
+  default) ships with proposal 0034's caller-metadata work.
+- **Span observation per node.** `answer_briefing` is the only node
+  here; a multi-node graph would produce a tree of nested Span
+  observations under the Trace.
+- **Generation observation per LLM call.** Carries `model`, `usage`,
+  `output`, and the prompt-identity metadata. In a production Langfuse
+  dashboard this is what the "Generation" detail view renders.
+- **`prompt_entity_link`** is the value `Prompt.observability_entities['langfuse_prompt']`
+  carried — a sentinel string in this demo, a real Langfuse SDK Prompt
+  object in production. When the backend doesn't surface the reference
+  (e.g., a filesystem backend), the link is absent but the
+  `metadata.prompt` map (name, version, label, hashes) still appears
+  for traceability.
+
+## Swapping to a real Langfuse SDK
+
+The observer's `client` parameter is `LangfuseClient`-Protocol-typed,
+so any structurally-compatible value works:
+
+```python
+from langfuse import Langfuse
+
+client = Langfuse(
+    public_key="pk-lf-...",
+    secret_key="sk-lf-...",
+    host="https://cloud.langfuse.com",
+)
+observer = LangfuseObserver(client=client, disable_llm_payload=False)
+```
+
+If the installed SDK version's `trace` / `span` / `generation` method
+signatures match the Protocol exactly, this is the whole change. If
+they diverge (renamed kwargs, return-type quirks), wrap the SDK in a
+small adapter class that implements `LangfuseClient` and delegates to
+the SDK call-by-call. The Protocol surface is narrow — four methods —
+so the adapter is on the order of 40 lines.
+
+**No specific `langfuse` SDK version is validated in CI as of this
+release.** The Protocol matches the SDK's documented low-level shape,
+but `langfuse` has shifted between major versions (v2 → v3 introduced
+API changes). A follow-on release pins a tested `[langfuse]` extras
+range and a runtime `isinstance(client, LangfuseClient)` check; until
+then, smoke-trace in your own environment with whichever `langfuse`
+version your stack already uses and write a thin adapter if any
+kwargs don't line up.
+
+For prompt linkage: in production, the
+`Prompt.observability_entities['langfuse_prompt']` value is the SDK's
+own Prompt-entity object (returned by `langfuse_client.get_prompt(...)`)
+rather than the sentinel string this demo uses. The observer passes
+that value straight through to the SDK's `generation(..., prompt=...)`
+argument, which is what the SDK uses to establish the native link.
+
+## Composition with OTel
+
+Both observers consume the same `NodeEvent` stream and can be attached
+together:
+
+```python
+graph.attach_observer(OTelObserver(span_processor=batch))
+graph.attach_observer(LangfuseObserver(client=langfuse_client))
+```
+
+Their `disable_llm_spans` / `disable_llm_payload` flags are
+independent. The `correlation_id` cross-cutting attribute is the join
+key — find a slow Generation in Langfuse, search for the
+`correlation_id` in OTel logs to see the surrounding infrastructure
+activity.