FE-1221: Add structured correlated logging across the Petrinaut optimization pipeline

[kube]

Closes FE-1221 (part of FE-1217).

Stacked on #9058 (base cf/fe-1220-...).

What

Structured, correlated logs across NodeAPI, the Python optimizer, and the Petrinaut CLI, with one correlation chain traceable end to end:

x-hash-request-id (NodeAPI) → run_id / X-Optimization-Run-ID (optimizer) → PETRINAUT_CORRELATION_ID (CLI spawn env) → correlationId in CLI stderr diagnostics.

NodeAPI (apps/hash-api, @local/petrinaut-optimizer-client)

• openPetrinautOptimizationStream gains a requestId option (sent as x-hash-request-id) and returns { events, optimizationRunId } (from the X-Optimization-Run-ID response header; also captured from failed responses).
• The handler builds a request-scoped logger.child({ requestId }) and logs request acceptance, validation failures (issue counts + bounded schema paths only), capacity rejections, stream-opened (with the optimizer run id), and a terminal outcome with durationMs — including the previously-silent client-disconnect case.

Optimizer service (apps/petrinaut-opt)

• New stdlib-only src/logging_config.py: JSON-lines formatter to stdout (service, correlation fields, HASH_PETRINAUT_OPT_LOG_LEVEL), idempotent config, charset-validated traceparent parsing.
• Endpoints read x-hash-request-id/traceparent; lifecycle events (admitted, capacity rejection, init started/failed, study started/completed/failed, per-trial outcomes, wall-clock ceiling, client disconnect, worker-join timeout, CLI termination path, slot release) log with run_id/request_id/trace_id.
• CLI stderr is no longer discarded: each line is forwarded to a pn_cli logger, bounded per-line (2000 chars), in pending memory (16 KiB), and per run (1000 lines, then drained silently).

Petrinaut CLI

• New src/runtime/diagnostics.ts: JSON-per-line stderr logger. The ready-handshake line remains the exact, unchanged first stderr line; subsequent lines are bootstrap_completed, one request line per handled request (id/method/durationMs/outcome — never params or error text), and stdin_eof.

Redaction

Logs never include the manifest, user-authored code, or raw request bodies. Error strings that can embed user identifiers (per-trial CLI errors, init-failure messages) are truncated and/or replaced with a stable classification in logs; the full message still reaches the requester in the HTTP response. Reviewed adversarially; the confirmed findings (unbounded aggregate stderr forwarding, raw error strings in logs, unbounded validation-path log field, dropped early CLI diagnostic, uncorrelated pre-stream failure, permissive traceparent parsing) are all fixed in this branch.

Tests

• apps/petrinaut-opt: 89 pytest
• apps/hash-api petrinaut-optimizer: 19 vitest
• @local/petrinaut-optimizer-client: 15 vitest
• libs/@hashintel/petrinaut-cli: 36 vitest

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
hash	Ignored		Jul 19, 2026 3:50am
hashdotdesign-tokens	Ignored		Jul 19, 2026 3:50am
petrinaut	Skipped		Jul 19, 2026 3:50am

[cursor]

PR Summary

Medium Risk
Touches the authenticated optimization proxy and optimizer admission path with new observability; logging is deliberately redacted but failure paths and stderr forwarding still need correct bounds in production.

Overview
Adds end-to-end correlated structured logging for Petrinaut optimization so NodeAPI, the Python optimizer, and the CLI can be joined in one chain: x-hash-request-id → optimizer run_id / X-Optimization-Run-ID → PETRINAUT_CORRELATION_ID on the CLI.

NodeAPI now uses a request-scoped child logger and logs acceptance, validation failures (counts and bounded paths only), capacity rejections, stream open, and terminal outcomes including client disconnect. It forwards x-hash-request-id upstream and reads optimizationRunId from the optimizer client (including on pre-stream HTTP errors). Validation path segments are capped before they appear in responses or logs.

petrinaut-opt adds JSON-lines stdout logging with request_id, trace_id (from validated traceparent), and run_id on lifecycle events. CLI stderr is forwarded into logs with per-line, buffer, and per-run caps. Init failures log a stable category, not raw CLI text that may embed user content.

Petrinaut CLI emits JSON stderr diagnostics after the unchanged ready line (bootstrap_completed, per-request summaries without params/errors, stdin_eof), driven by PETRINAUT_CORRELATION_ID from the parent spawn env only.

Docs and tests assert redaction (no manifests, user code, or unbounded user-controlled strings in logs).

^{Reviewed by Cursor Bugbot for commit 2f81742. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 2f81742. Configure here.}

[cursor]

Terminal log marks failures completed

Medium Severity

After forwardOptimizationEvents returns, the handler always logs Petrinaut optimization finished with outcome: "completed". That path runs even when the last forwarded NDJSON event was type: "error" from upstream SSE, so failed studies are recorded as completed in structured logs.

 

^{Reviewed by Cursor Bugbot for commit 2f81742. Configure here.}