feat(cli): add --log-file for JSONL run logs#534
Open
davidslater wants to merge 2 commits into
Open
Conversation
Implements issue #380. Adds a --log-file flag (and THREAT_DETECTION_LOG_FILE env var) that writes a structured JSON Lines trace of a detection run. - New pkg/runlog package: a nil-safe JSONL logger whose records lead with time/level/event and emit remaining fields in deterministic sorted order; Open creates a 0600, truncated file the logger owns. - Wire logging into cmd/threat-detect run(): run_start, artifacts_loaded, prompt_built, per-attempt events, verdict, detection_failed, and a terminal status record mirroring the stderr status line (reason + exit code). - Log file is additive observability only: result JSON contract and exit codes are unchanged; failure to open the log file is a config error. - Docs: README CLI flag + JSONL section, spec TD-20a, CLAUDE.md flags/layout. - Tests for record shape, key ordering, reserved-key shadowing, nil logger, and file permissions/truncation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: David Slater <12449447+davidslater@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds optional JSONL run logging for detection observability without changing verdicts or exit codes.
Changes:
- Adds a nil-safe JSONL logger and tests.
- Wires
--log-fileand its environment variable into detection runs. - Documents the feature and normative contract.
Show a summary per file
| File | Description |
|---|---|
cmd/threat-detect/main.go |
Adds CLI configuration and run-event logging. |
pkg/runlog/runlog.go |
Implements the JSONL writer. |
pkg/runlog/runlog_test.go |
Tests formatting and file behavior. |
specs/threat-detection-spec.md |
Defines TD-20a. |
README.md |
Documents usage and output. |
CLAUDE.md |
Updates repository guidance. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 6/6 changed files
- Comments generated: 6
- Review effort level: Medium
Comment on lines
+147
to
+152
| logger.Info("run_start", map[string]any{ | ||
| "version": detector.Version, | ||
| "engine": engineID, | ||
| "model": model, | ||
| "retries": retries, | ||
| }) |
| // Open the JSONL run log if requested. A failure here is a config error: | ||
| // the caller explicitly asked for logs and should learn they were not written. | ||
| if logFile != "" { | ||
| l, err := runlog.Open(logFile) |
Comment on lines
+50
to
+54
| f, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o600) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("opening JSONL log file: %w", err) | ||
| } | ||
| return &Logger{w: f, closer: f, now: time.Now}, nil |
| } | ||
| l.mu.Lock() | ||
| defer l.mu.Unlock() | ||
| _, _ = l.w.Write(line) |
| `THREAT_DETECTION_LOG_FILE` environment variable). When enabled, the detector MUST | ||
| write one JSON object per line, each containing at least the `time`, `level`, and | ||
| `event` keys, and MUST record a terminal `status` event carrying the same `reason` | ||
| and exit `code` as the stderr status line. The run log is an additive observability |
| flag.StringVar(&model, "model", "", "Model to use for detection") | ||
| flag.StringVar(&promptFile, "prompt-template", "", "Path to custom prompt template (defaults to built-in)") | ||
| flag.StringVar(&outputJSON, "output", "", "Path to write JSON result (defaults to stdout)") | ||
| flag.StringVar(&logFile, "log-file", os.Getenv("THREAT_DETECTION_LOG_FILE"), "Path to write JSONL run logs (env: THREAT_DETECTION_LOG_FILE)") |
This was referenced Jul 14, 2026
- Normalize engine ID before logging run_start so the audit record shows the engine that actually runs (copilot) instead of "". Adds exported engine.Canonical/DefaultEngineID and reuses them in engine.New. - Reject --log-file colliding with --output (cleaned/symlink-equivalent paths) as a config error to avoid corrupting both sinks. - runlog.Open now chmods the descriptor to 0600 so truncating a pre-existing 0644 file cannot leave audit data world-readable. - runlog now records the first encode/write error and surfaces it from Close so disk-full/short-write failures are not silently lost. - Spec TD-20a: rename the status field from `code` to `exit` to match the implementation and README. - Add integration tests for run(): JSONL log contents/lifecycle/status, env-var resolution, path-collision rejection, and log-open failure; plus runlog chmod/write-error tests and engine.Canonical tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: David Slater <12449447+davidslater@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements #380 ("Jsonl logs — Add flag to write jsonl logs to file").
Adds a
--log-file <path>flag (also configurable via theTHREAT_DETECTION_LOG_FILEenvironment variable) that writes a structured JSON Lines trace of a detection run — one JSON object per line — to a file. This is an additive observability sink for debugging/auditing detection runs in CI; it does not change the result JSON contract or exit codes.What's included
pkg/runlogpackage — a small, nil-safe JSONL logger.time(RFC 3339),level(info/error), andevent, followed by event-specific fields in deterministic sorted order (stable, diff-friendly).Open()creates a fresh0600file (truncating any existing one) that the logger owns and closes.nil *Loggeris a valid no-op, so call sites stay clean when logging is disabled.cmd/threat-detect/main.go— emitsrun_start,artifacts_loaded,prompt_built, per-attempt events (attempt_start/attempt_recorded/attempt_no_verdict),verdict,detection_failed, and a terminalstatusrecord carrying the samereasonandexitcode as the existing stderr status line.Example output
{"time":"2026-07-14T18:00:00Z","level":"info","event":"run_start","engine":"copilot","model":"","retries":1,"version":"1.2.3"} {"time":"2026-07-14T18:00:03Z","level":"info","event":"verdict","has_threats":false,"malicious_patch":false,"prompt_injection":false,"reasons":[],"secret_leak":false} {"time":"2026-07-14T18:00:03Z","level":"info","event":"status","exit":0,"reason":"result_recorded"}Testing
make fmt lint build testcould not be run locally. Changes were reviewed manually for compile-correctness and gofmt (tab) compliance. Please runmake agent-finishin CI / locally to validate before merge.Closes #380.