Skip to content

Commit 6569519

Browse files
feat: structured logging + audit trail (W2-6) (#51)
## What changed & why Implements W2-6 (`.claude/tasks/prod-readiness/w2-6-observability.md`): replaces ad-hoc `println` diagnostics with `Logging`-based structured logging carrying per-chat-turn correlation ids, and wires the previously-dormant VeriSimDB audit-trail path into the tool-execution runtime. - **`src/tools/observability.jl`** (new) — the observability module: - `configure_logging!()` installs a stderr-backed `Logging.ConsoleLogger` whose level is controlled by `STATISTIKLES_LOG_LEVEL` (`Debug`/`Info`/`Warn`/`Error`, default `Info`; unrecognized values fall back to `Info` rather than silently going to `Debug`-volume output). Called from `__init__` so it's honoured even from a precompiled image, unlike the `const`-baked `STATISTIKLES_LM_URL`. - `new_correlation_id()` / `tool_call_identifier()` — UUID-based per-turn and per-tool-call ids (the latter reuses the LLM-assigned `tool_call["id"]` when present, mints a fresh one otherwise). - `hash_arguments()` / `summarize_result()` — never log or persist raw arguments/results (may contain user data): only an order-independent argument hash and a shape-only summary (type, keys, length) ever leave the process. - `log_tool_call()` — the single structured log record per tool call (`@info` on success, `@warn` on an error-shaped result): correlation id, tool-call id, tool name, arg hash, result shape summary, duration in ms. Replaces the previous `println("[symbolic] executing: ...")` breadcrumb in `lmstudio.jl`. - `build_audit_record()` / `record_audit!()` — construct the VeriSimDB audit record (turn id, tool-call id, tool, arg hash, result provenance, timestamp) unconditionally and testably; only *persist* it (via `store_result(..., modality="audit")` from `src/bridge/verisimdb_schema.jl`) when `STATISTIKLES_AUDIT_PERSIST` is opted in (default off). Persistence failures are caught and logged, never thrown, so a down/absent VeriSimDB backend never breaks a chat turn. - **`src/tools/lmstudio.jl`** — `process_tool_calls` now accepts/threads a `correlation_id` (minted fresh if the caller omits one), times each tool call, and emits both the structured log record and the audit record for every tool call instead of the removed `println` breadcrumb. Returns `correlation_id` in its NamedTuple so callers can propagate it. - **`src/tools/chat.jl`** — mints one `correlation_id` per chat turn, forwards it to `process_tool_calls` and into the guardrail retry path in `enforce_numeric_boundary!`, and logs `lm_studio_call_failed` / `chat_turn_error` with that id instead of printing errors directly to stdout. Human-facing REPL output (prompts, answers) stays on stdout via `println` and is never routed through the logger, so log lines never interleave with a conversation. - **`src/tools/executor.jl`** — `execute_tool`'s exception path now also emits an `@debug`-level structured breadcrumb (tool name + arg hash, never raw arguments; off by default, opt in with `STATISTIKLES_LOG_LEVEL=Debug`). - **`src/Statistikles.jl`** — `using Logging`, includes `tools/observability.jl` first among the interface-layer files, adds `__init__() = configure_logging!()`. - **`Project.toml`** — adds the `Logging` stdlib dependency (`[deps]` + `[compat]`). - **`QUICKSTART-MAINTAINER.adoc`** — documents `STATISTIKLES_LOG_LEVEL` and `STATISTIKLES_AUDIT_PERSIST` alongside the existing LM Studio env vars. - **`test/observability_test.jl`** (new) — unit tests for correlation-id generation, argument hashing (order-independence, no raw-value leakage), result summarization, audit-record construction (success/error shape, optional `duration_s`), `audit_persistence_enabled()` truthy/falsy parsing, log-level parsing, and `configure_logging!()` honouring `STATISTIKLES_LOG_LEVEL`. Structured log records are captured with `Test.collect_test_logs`/`with_logger` (correlation id + tool metadata asserted, no raw values) — including an end-to-end `process_tool_calls` check that the same `correlation_id` threads through every tool-call log record for a turn, and that two independent calls without an explicit id get distinct ones. No live VeriSimDB backend or LLM endpoint is required anywhere in this file. - **`test/runtests.jl`** — wires in `observability_test.jl`. This closes the gap called out in the work order: the runtime path previously had only bare `println`/stdout error printing and never invoked the VeriSimDB persistence layer at all, despite the product's auditability claims. ## Verification Reviewer approved this task with `must_fix: (none)`, so no fixes were required before finalizing. Re-ran the full suite fresh to confirm green under WSL Debian, Julia 1.10.11, serialized with `flock`: ``` flock /tmp/statistikles-julia.lock -c 'cd <repo> && julia --project=. -e "using Pkg; Pkg.test()"' ``` Result: **all green**, exit code 0. | Testset | Pass / Total | |---|---| | Statistikles Full Test Suite | 424 / 424 | | Statistikles E2E Pipeline Tests | 155 / 155 | | Statistikles Property-Based Tests | 3800 / 3800 | | Reference Validation (ground truth) | 62 / 62 | | Degenerate Input Guards | 943 / 943 | | Neural-Boundary Guardrail | 80 / 80 | | Executor Router Coverage | 131 / 131 | | Reference Validation — Advanced Modules | 42 / 42 | | **Observability (new)** | **73 / 73** | Total: 5710 / 5710 passing. The known unseeded-`randn()` MANOVA flake (separate in-flight PR) did not occur on this run — no rerun was needed. ## Skipped / out of scope Per the work order: standing up a real VeriSimDB backend and log-shipping/telemetry infra are explicitly out of scope. Audit persistence remains pluggable and off by default (`STATISTIKLES_AUDIT_PERSIST=false`); only record *construction* is exercised in CI, matching requirement 3's "make persistence pluggable/no-op-by-default" guidance. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 95d867b commit 6569519

0 file changed

File tree

    0 commit comments

    Comments
     (0)