feat(itmux): reusable observability exporter primitive#256
feat(itmux): reusable observability exporter primitive#256NeuralEmpowerment wants to merge 126 commits into
Conversation
|
Finish-line refresh pushed in 150b65e. What changed:
Local validation before push:
Notes:
|
|
Acknowledging delegated review finding: the prior PR body overstated the current code by describing an active |
|
Refactor follow-up for the main.rs size finding is pushed in 5a2589d. The LangFuse read/score CLI implementation and tests are now in providers/workspaces/interactive-tmux/driver-rs/src/langfuse.rs; main.rs is back down to CLI declarations/dispatch plus the workspace/run flows. Validation: cargo fmt, git diff --check, and cargo test --manifest-path providers/workspaces/interactive-tmux/driver-rs/Cargo.toml all passed locally. Cargo still prints the pre-existing APS template-manifest warnings from the dependency checkout before compiling, but itmux compiled and all tests passed. |
|
Follow-up pushed in 2f5d96e.\n\nChanges:\n- Fixed LangFuse score readback robustness: Rust CLI and MCP direct fallback now send one narrowing backend filter, then enforce trace/name/score-id/type locally. This fixes the live local case where trace_id+name returned zero even though trace-only/name-only/score-id reads worked.\n- Added coverage for single-backend-filter score URL behavior.\n- Added scripts/langfuse-local.sh health for central-server reachability checks.\n- Updated the runbook with a concrete Mac mini/Tailscale pattern using tag:langfuse and tag:agents, plus client health check command.\n\nValidation:\n- live local combined score readback now returns the expected Codex score\n- git diff --check\n- full cargo test --manifest-path providers/workspaces/interactive-tmux/driver-rs/Cargo.toml\n- python3 -m py_compile plugins/observability/mcp/langfuse_server.py\n- python3 plugins/observability/mcp/langfuse_server.py --self-test\n- bash -n scripts/langfuse-local.sh scripts/langfuse-observability-doctor.sh\n- scripts/langfuse-local.sh health |
1 participant
Summary
Adds the reusable observability primitive on top of the
itmux runcontract and finalizes the LangFuse integration boundary after the ADR-039 pivot.Current code in this PR provides:
ObservabilityExporterfanout with activefileandsyntropic_jsonlbackendsObservabilityBundleexporter reports with status, event counts, targets, links, and errorsitmux run,itmux codex-exec, anditmux claude-transcriptlocal observability export pathsitmux run --codex-mode execfor Codex recipes that need structured tool/token/cost telemetry through the standard run surfaceitmux langfuse-tracesummaries, including compact generation-level cost breakdowns and--output summaryitmux langfuse-tracesdiscovery for recent runs with harness/provider/model/cost/observation countsitmux langfuse-score/itmux langfuse-scoresfeedback write/read for trace-scoped learning-loop scoresitmux langfuse-trace --include-scoresfor one-call trace telemetry plus feedback-score retrospectivesImportant boundary: this PR no longer contains a native Rust LangFuse OTLP write exporter. Historical experiments in this branch proved why the direct Rust OTLP path was not the right rich-trace UX. ADR-039 supersedes that path. LangFuse writes for Claude/Codex rich traces happen through LangFuse's official Claude Code and Codex plugins; Agentic Primitives owns local JSONL/Syntropic fanout plus LangFuse query/score/readiness tooling.
Current Evidence
Current local proof is centered on the official-plugin path and agent query/readiness tooling:
0e553fc833c71639acd03be9807eb616b3d2561d7c0557c12fd427c02a16e2f3itmux langfuse-tracecan query both traces and expose harness/provider/model/tool/cost summariesagentic-langfuseMCP server can query the same summaries for learning-loop usescripts/langfuse-local.sh smokeloads ignored local env and runs the setup/readiness doctorscripts/langfuse-observability-doctor.shreports Claude/Codex official plugin readiness, runtime env set/missing state, local JSONL/Syntropic fanout support, MCP presence, and confirms the direct writer is removedHistorical evidence that remains useful but is no longer the active public setup path:
langfuse_otlpand--observability-langfuseexperiments underexperiments/2026-07-07--langfuse-*Current Boundary
.6is represented by the reusable observability substrate in this PR:.9is not closed by this PR. It is unblocked for target deployment work:itmux langfuse-*and MCP learning-loop readsCodex and Claude both have rich observability paths now:
--observability-fileand--observability-syntropic-fileitmux langfuse-trace(s),itmux langfuse-score(s), andagentic-langfuseMCPThis PR remains draft because it is stacked on
feat/itmux-run-contract/ PR #247 rather than directly onmain.Validation
Local checks for the latest head include:
bash -n scripts/langfuse-local.sh scripts/langfuse-observability-doctor.shgit diff --checkcargo test --manifest-path providers/workspaces/interactive-tmux/driver-rs/Cargo.tomlscripts/langfuse-local.sh smokeGitHub checks are green for the latest head
150b65e, including:itmux (fmt, clippy, test)claude-cliclaude-cliinteractive-tmuxKnown caveat: cargo may print pre-existing APSS template
{{slug}}package-name diagnostics before compiling; those diagnostics are non-fatal.Review Notes
A delegated review correctly flagged that the old PR body overstated the current code by claiming an active
langfuse_otlpwriter backend and real native Rust OTLP export. That claim is stale and has been corrected here. The current code intentionally has onlyfileandsyntropic_jsonlwrite exporters in the active run contract.