Skip to content

Commit 9422e88

Browse files
willwashburnclaude
andauthored
feat(ingest): single-transcript Claude hook fast-path + global --quiet (#420)
* feat(ingest): single-transcript Claude hook fast-path + global --quiet Restores 1.x parity gaps called out in #375: - `relayburn-sdk`: new `ingest_claude_transcript_path` verb that parses just the one JSONL the caller hands it and persists an EOF cursor so a follow-up `ingest_all` skips it. `ingest_claude_session` now delegates through it. - `relayburn-cli`: `burn ingest --hook claude` drives the new fast-path on the payload's `transcript_path`, bounding per-hook cost to one parse. Falls back to a full sweep if the payload omits the path. - `relayburn-cli`: `--quiet` is no longer hook-only — accepted in default (one-shot) and `--watch` modes too. One-shot mode still writes its final summary on stdout so pipelines stay capturable. - `relayburn-cli`: documents the intentional non-port of 1.x's `--opencode-stream` / `--opencode-url` / `--opencode-global` flags (the file-based OpenCode adapter + FS-event watch is the supported 2.x path; the SDK still ships `OpencodeStreamIngestor` for embedders who want to consume the SSE feed themselves). Tests: - SDK: `ingest_claude_transcript_path_*` covers the EOF cursor contract and the missing-file no-op policy. - CLI smoke: one-shot stdout summary, `--quiet` keeps stdout summary, hook fast-path no-op on missing transcript, hook payload validation, `--quiet` suppresses the hook payload warning, watch+hook mutex. Closes #375. https://claude.ai/code/session_011ubB69Zxijqb1BsYVYL9iQ * fix(ingest): wire hook payload fallback for missing transcript_path The previous guard returned 0 whenever transcript_path was absent, making the documented ingest_all fallback unreachable. Older Claude Code hook payloads carry session_id without transcript_path; they now fall through to a full sweep instead of being ignored. Add a smoke test pinning the new contract: a payload with session_id but no transcript_path runs under --quiet without stderr noise. Caught by Codex review on #420. https://claude.ai/code/session_011ubB69Zxijqb1BsYVYL9iQ * fix(ingest): honor --quiet in watch mode + propagate metadata errors Three reviewer-flagged fixes: - `relayburn-cli`: `run_watch` now gates the TaskProgress spinner and all subsequent set_task / suspend / finish_and_clear calls behind Option<TaskProgress>, matching the run_once / run_hook pattern. With `--watch --quiet` the spinner is no longer created at all. - `relayburn-sdk`: `ingest_claude_transcript_path` only suppresses NotFound when stat-ing the transcript; permission / IO errors now propagate to the caller instead of being silently dropped as an empty report. - `crates/relayburn-cli/tests/smoke.rs`: pin the one-shot quiet contract harder — stderr must be empty, not just "stdout still has the summary". - `CHANGELOG.md`: drop the trailing "Closes #375" per the impact-first style guide. https://claude.ai/code/session_011ubB69Zxijqb1BsYVYL9iQ * fix(ingest): resolve pending stamps + silence missing transcript in fast-path Two reviewer-flagged gaps in ingest_claude_transcript_path: - The fast-path was saving the EOF cursor without calling resolve_pending_stamps_for_session_in, so pending stamps for the session went unapplied and IngestReport.applied_pending_stamps was always 0. Mirror the per-harness sweep's candidate construction (using session_id + project from result.turns[0]) and delegate to resolve_pending_stamps_for_report before persisting the cursor. - The NotFound arm wrote a "[burn] no session file found at ..." line to stderr, contradicting the documented hook policy of staying silent so the caller decides whether to surface the rotation. Drop the eprintln; keep the empty-report return. https://claude.ai/code/session_011ubB69Zxijqb1BsYVYL9iQ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 7660ef5 commit 9422e88

9 files changed

Lines changed: 422 additions & 94 deletions

File tree

CHANGELOG.md

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,19 @@ Cross-package release notes for relayburn. Package changelogs contain package-le
66

77
### Added
88

9+
- `relayburn-sdk`: `ingest_claude_transcript_path(ledger, path, opts)`
10+
per-transcript Claude fast-path used by `burn ingest --hook claude` so
11+
the hook ingests only the one JSONL the payload points at instead of a
12+
full sweep.
13+
14+
### Changed
15+
16+
- `relayburn-cli`: `burn ingest --hook claude` now drives the new
17+
single-transcript fast-path, bounding per-hook cost to one JSONL parse.
18+
- `relayburn-cli`: `burn ingest --quiet` is now accepted in default
19+
(one-shot) and `--watch` modes (no longer hook-only). Suppresses the
20+
progress spinner, watch banner, and per-tick summaries; one-shot mode
21+
still writes its final summary line to stdout for pipeline capture.
922
- `relayburn-sdk`: `Inference` aggregate keys per-API-call rollups by
1023
`(source, session_id, request_id)` with merged usage and `kind`
1124
(`reasoning` / `message` / `tool-use` / `mixed`). Read via
@@ -27,9 +40,6 @@ Cross-package release notes for relayburn. Package changelogs contain package-le
2740
- `burn summary`: new `subagents: X paired, Y orphan` line (and matching `subagents` key in `--json`) populated by a lazy walk over `~/.claude/projects/`. Skipped entirely when no sidecars exist anywhere reachable so pre-#435 outputs stay byte-identical. Honors `BURN_CLAUDE_PROJECTS_DIR` for test sandboxing. (#435)
2841
- Ledger schema bumped to v4 — new nullable `turns.subagent_id TEXT` column denormalizes `TurnRecord.subagent.agent_id` so subagent rows are queryable without re-deserializing `record_json`. Migrated in place by `ALTER TABLE … ADD COLUMN`; pre-v4 rows stay `NULL` and are backfilled by `burn state rebuild`. (#435)
2942
- `relayburn-sdk`: Claude slash-command triads (`/review`, `/init`, custom skills) now collapse into one synthetic `Skill` activity instead of inflating the activity count three rows at a time. Detection pins on the caveat → invocation → stdout parent-UUID chain shape with `<command-name>` / `<local-command-stdout>` purpose checks, so real user prompts that happen to look structurally similar are not misdetected. Token attribution stays on the underlying assistant rows — `Skill` is a view, not a billing reattribution. New `ActivityCategory::Skill` variant and `detect_slash_triads` helper. (#438)
30-
31-
### Changed
32-
3343
- `relayburn-sdk`: Claude Code parser now skips harness-injected
3444
`<task-notification>` rows when emitting `UserTurnRecord`s. The detector
3545
matches shape AND purpose across three envelope variants

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -154,8 +154,8 @@ harness spawn. Default mode scans Claude Code, Codex, and OpenCode stores once.
154154
|---|---|
155155
| `--watch` | Keep polling session stores in the foreground. |
156156
| `--interval <ms>` | Poll interval in milliseconds. Default: `1000`. |
157-
| `--quiet` | Suppress normal ingest messages. |
158-
| `--hook claude` | Read one Claude Code hook payload from stdin and ingest its transcript. |
157+
| `--quiet` | Suppress stderr progress spinner / breadcrumbs. One-shot mode still writes the final summary on stdout. |
158+
| `--hook claude` | Read one Claude Code hook payload from stdin and ingest its single transcript via the SDK fast-path. |
159159

160160
| Example | Result |
161161
|---|---|

crates/relayburn-cli/src/cli.rs

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -128,10 +128,12 @@ pub enum Command {
128128
/// - No flags: scan all known session stores once and exit.
129129
/// - `--watch` (optionally with `--interval <MS>`): foreground poll loop
130130
/// driven by [`relayburn_sdk::start_watch_loop`].
131-
/// - `--hook <HARNESS> [--quiet]`: stdin-driven hook entrypoint. Today
132-
/// only `--hook claude` is supported; the `--quiet` flag suppresses
133-
/// non-error stderr breadcrumbs so it is safe to call from every
134-
/// Claude Code hook.
131+
/// - `--hook <HARNESS>`: stdin-driven hook entrypoint. Today only
132+
/// `--hook claude` is supported.
133+
///
134+
/// `--quiet` is accepted in every mode and suppresses non-error stderr
135+
/// breadcrumbs (progress spinner, watch banner, hook summary). One-shot
136+
/// mode still writes its final summary line to stdout regardless.
135137
///
136138
/// `--watch` and `--hook` are mutually exclusive; the presenter rejects
137139
/// the combination at runtime with exit 2 (matching TS).
@@ -153,11 +155,13 @@ pub struct IngestArgs {
153155
#[arg(long, value_name = "HARNESS")]
154156
pub hook: Option<String>,
155157

156-
/// Suppress non-error stderr breadcrumbs. Used by hook callers so
157-
/// the surrounding tool invocation isn't blocked by a noisy
158-
/// pipeline. Only meaningful with `--hook`; clap rejects `--quiet`
159-
/// on its own (or with `--watch`) so a typo can't silently no-op.
160-
#[arg(long, requires = "hook")]
158+
/// Suppress non-error stderr breadcrumbs (progress spinner, watch
159+
/// banner, and per-tick `[burn] ingest: …` summaries). Useful for
160+
/// hook callers so the surrounding tool invocation isn't blocked by
161+
/// a noisy pipeline, and for automation/cron drivers of `--watch` /
162+
/// one-shot mode. One-shot mode still writes its final summary line
163+
/// to stdout so pipelines can capture it.
164+
#[arg(long)]
161165
pub quiet: bool,
162166

163167
/// Force the polling driver in `--watch` mode instead of the

crates/relayburn-cli/src/commands/ingest.rs

Lines changed: 107 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -14,13 +14,11 @@
1414
//! / SIGTERM.
1515
//! - `--hook claude` = `runIngestHook` — stdin-driven hook payload.
1616
//! Today only `--hook claude` is wired here (Codex / OpenCode hooks
17-
//! were never part of the TS surface either). The hook path
18-
//! currently ingests via a full `ingest_all` sweep, since the SDK
19-
//! does not yet expose a single-transcript verb. Practically this
20-
//! is no slower than the TS hook because Claude hooks fire at
21-
//! session-end and the sweep short-circuits on unchanged cursors;
22-
//! the cost is bounded by the number of new sessions, not by the
23-
//! hook payload.
17+
//! were never part of the TS surface either). The hook path uses
18+
//! the SDK's single-transcript fast-path
19+
//! (`ingest_claude_transcript_path`) on the `transcript_path` from
20+
//! the Claude Code hook payload so the per-call cost is bounded by
21+
//! the one JSONL file, not by the number of sessions on disk.
2422
//!
2523
//! Output shape: every successful run writes a single
2624
//! `[burn] ingest: ingested N session(s) (+M turn(s))` line. The
@@ -29,16 +27,28 @@
2927
//! `packages/cli/src/commands/ingest.ts:121-126`); `--watch` and
3028
//! `--hook` modes log on **stderr** so the foreground banner / hook
3129
//! breadcrumbs don't pollute downstream stdout consumers. `--quiet`
32-
//! (only valid with `--hook`) suppresses the hook breadcrumb when the
33-
//! report is empty.
30+
//! is accepted in every mode: in `--watch` and `--hook` it silences
31+
//! every breadcrumb; in one-shot mode it suppresses the progress
32+
//! spinner / gap warnings but the final stdout summary still prints.
33+
//!
34+
//! Not ported from 1.x: `--opencode-stream`, `--opencode-url`, and
35+
//! `--opencode-global`. The 1.x stream subscribed to the OpenCode dev
36+
//! server's SSE feed directly. 2.x consumes OpenCode through the
37+
//! file-based session store (`~/.local/share/opencode/storage`) on
38+
//! every sweep, and `--watch` reacts to FS events the moment OpenCode
39+
//! writes a new event file. The `OpencodeStreamIngestor` parser still
40+
//! lives in `relayburn-sdk::reader::opencode_stream` for embedders
41+
//! that want to consume the SSE feed themselves, but the CLI no
42+
//! longer ships HTTP client wiring for it.
3443
3544
use std::io::{self, Read};
45+
use std::path::PathBuf;
3646
use std::sync::Arc;
3747
use std::time::Duration;
3848

3949
use relayburn_sdk::{
40-
default_session_roots, ingest_all, start_watch_loop, IngestReport, IngestRoots, Ledger,
41-
LedgerHandle, LedgerOpenOptions, StartWatchLoopOptions,
50+
default_session_roots, ingest_all, ingest_claude_transcript_path, start_watch_loop,
51+
IngestReport, IngestRoots, Ledger, LedgerHandle, LedgerOpenOptions, StartWatchLoopOptions,
4252
};
4353

4454
use crate::cli::{GlobalArgs, IngestArgs};
@@ -78,23 +88,36 @@ pub fn run(globals: &GlobalArgs, args: IngestArgs) -> i32 {
7888
///
7989
/// Summary line is emitted on **stdout** (matching TS `runIngestOnce`
8090
/// at `packages/cli/src/commands/ingest.ts:121-126`) so callers can
81-
/// capture pipeline output without redirecting stderr.
91+
/// capture pipeline output without redirecting stderr. `--quiet`
92+
/// suppresses the progress spinner and gap-warning side channel but
93+
/// leaves the final stdout summary alone so pipeline consumers always
94+
/// see the report.
8295
fn run_once(globals: &GlobalArgs, quiet: bool) -> i32 {
83-
let _ = quiet; // `--quiet` is hook-only (clap `requires = "hook"`); kept in
84-
// the dispatch signature for symmetry with run_watch / run_hook.
85-
let progress = TaskProgress::new(globals, "ingest");
86-
progress.set_task("opening ledger");
96+
let progress = (!quiet).then(|| {
97+
let p = TaskProgress::new(globals, "ingest");
98+
p.set_task("opening ledger");
99+
p
100+
});
87101
let mut handle = match open_handle(globals) {
88102
Ok(h) => h,
89103
Err(err) => {
90-
progress.finish_and_clear();
104+
if let Some(p) = &progress {
105+
p.finish_and_clear();
106+
}
91107
return report_error(&err, globals);
92108
}
93109
};
94-
progress.set_task("scanning sessions");
95-
let opts = progress.ingest_options(globals.ledger_path.clone());
110+
if let Some(p) = &progress {
111+
p.set_task("scanning sessions");
112+
}
113+
let opts = match &progress {
114+
Some(p) => p.ingest_options(globals.ledger_path.clone()),
115+
None => TaskProgress::quiet_ingest_options(globals.ledger_path.clone()),
116+
};
96117
let result = ingest_all(handle.raw_mut(), &opts);
97-
progress.finish_and_clear();
118+
if let Some(p) = &progress {
119+
p.finish_and_clear();
120+
}
98121
match result {
99122
Ok(report) => {
100123
log_report_oneshot(&report);
@@ -123,29 +146,38 @@ fn run_watch(globals: &GlobalArgs, args: &IngestArgs) -> i32 {
123146
None => 1000,
124147
};
125148

126-
let progress = TaskProgress::new(globals, "ingest");
127-
progress.set_task("opening ledger");
149+
let quiet = args.quiet;
150+
let progress = (!quiet).then(|| {
151+
let p = TaskProgress::new(globals, "ingest");
152+
p.set_task("opening ledger");
153+
p
154+
});
128155
let handle = match open_handle(globals) {
129156
Ok(h) => h,
130157
Err(err) => {
131-
progress.finish_and_clear();
158+
if let Some(p) = &progress {
159+
p.finish_and_clear();
160+
}
132161
return report_error(&err, globals);
133162
}
134163
};
135164

136-
progress.set_task("starting watcher");
165+
if let Some(p) = &progress {
166+
p.set_task("starting watcher");
167+
}
137168
let rt = match tokio::runtime::Builder::new_current_thread()
138169
.enable_all()
139170
.build()
140171
{
141172
Ok(rt) => rt,
142173
Err(err) => {
143-
progress.finish_and_clear();
174+
if let Some(p) = &progress {
175+
p.finish_and_clear();
176+
}
144177
return report_error(&err, globals);
145178
}
146179
};
147180

148-
let quiet = args.quiet;
149181
let no_fsevents = args.no_fsevents;
150182
// The FS-event driver may silently demote to polling at startup
151183
// (no watchable path yet) or mid-run (notify channel closes), so
@@ -156,9 +188,9 @@ fn run_watch(globals: &GlobalArgs, args: &IngestArgs) -> i32 {
156188
} else {
157189
"watching (FS events with polling fallback); Ctrl-C to stop".to_string()
158190
};
159-
if !quiet {
160-
if progress.is_visible() {
161-
progress.set_task(watch_message.clone());
191+
if let Some(p) = &progress {
192+
if p.is_visible() {
193+
p.set_task(watch_message.clone());
162194
} else if no_fsevents {
163195
eprintln!(
164196
"[burn] ingest: foreground ingest polling every {interval_ms}ms; Ctrl-C to stop",
@@ -184,15 +216,18 @@ fn run_watch(globals: &GlobalArgs, args: &IngestArgs) -> i32 {
184216
let ledger_home = ledger_home.clone();
185217
let watch_message = watch_message_for_ingest.clone();
186218
Box::pin(async move {
187-
progress.set_task("scanning sessions");
219+
if let Some(p) = &progress {
220+
p.set_task("scanning sessions");
221+
}
188222
let mut guard = h.lock().await;
189-
let opts = if quiet {
190-
TaskProgress::quiet_ingest_options(ledger_home)
191-
} else {
192-
progress.ingest_options(ledger_home)
223+
let opts = match &progress {
224+
Some(p) => p.ingest_options(ledger_home),
225+
None => TaskProgress::quiet_ingest_options(ledger_home),
193226
};
194227
let result = ingest_all(guard.raw_mut(), &opts);
195-
progress.set_task(watch_message);
228+
if let Some(p) = &progress {
229+
p.set_task(watch_message);
230+
}
196231
result
197232
})
198233
});
@@ -202,26 +237,29 @@ fn run_watch(globals: &GlobalArgs, args: &IngestArgs) -> i32 {
202237
// Match TS: only log a summary when the tick actually
203238
// appended turns. Empty ticks would otherwise drown the
204239
// user with zero-progress lines.
205-
if !quiet && report.appended_turns > 0 {
206-
progress_for_report.suspend(|| {
207-
eprint!("{}", render_ingest_line(report));
208-
});
240+
if let Some(p) = &progress_for_report {
241+
if report.appended_turns > 0 {
242+
p.suspend(|| {
243+
eprint!("{}", render_ingest_line(report));
244+
});
245+
}
209246
}
210247
});
211248

212249
let progress_for_error = progress_for_loop.clone();
213-
let on_error: relayburn_sdk::ErrorSink = Arc::new(move |err: &anyhow::Error| {
214-
progress_for_error.suspend(|| {
250+
let on_error: relayburn_sdk::ErrorSink = Arc::new(move |err: &anyhow::Error| match &progress_for_error {
251+
Some(p) => p.suspend(|| {
215252
eprintln!("[burn] ingest: {err}");
216-
});
253+
}),
254+
None => eprintln!("[burn] ingest: {err}"),
217255
});
218256

219257
// Default to the `notify`-backed FS-event driver against the
220258
// three session-store roots ingest scans. Falls back to polling
221259
// automatically when no path exists yet (fresh install) or
222260
// when the user passes `--no-fsevents`. The slow polling
223261
// backstop in the SDK keeps progress on filesystems where FS
224-
// events are unreliable. Closes #250.
262+
// events are unreliable.
225263
let watch_paths = default_session_roots(&IngestRoots::default());
226264
let opts = StartWatchLoopOptions::new(ingest_fn)
227265
.with_interval(Duration::from_millis(interval_ms))
@@ -235,7 +273,9 @@ fn run_watch(globals: &GlobalArgs, args: &IngestArgs) -> i32 {
235273
wait_for_stop_signal().await;
236274
controller.stop().await;
237275
});
238-
progress.finish_and_clear();
276+
if let Some(p) = &progress {
277+
p.finish_and_clear();
278+
}
239279

240280
0
241281
}
@@ -247,6 +287,12 @@ fn run_watch(globals: &GlobalArgs, args: &IngestArgs) -> i32 {
247287
/// non-zero exit can block the surrounding tool call); the Rust port
248288
/// keeps that policy — every error is logged to stderr but the exit
249289
/// code is `0` so the calling Claude Code session continues.
290+
///
291+
/// Fast-path: when the payload carries a `transcript_path` we drive
292+
/// the SDK's single-transcript verb against just that JSONL file
293+
/// instead of a full sweep. Falls back to `ingest_all` when the
294+
/// payload is missing `transcript_path` (older Claude Code releases
295+
/// occasionally elide it) so we still make forward progress.
250296
fn run_hook(globals: &GlobalArgs, hook: &str, quiet: bool) -> i32 {
251297
if hook != "claude" {
252298
eprintln!("burn: unsupported hook harness: {hook}");
@@ -267,34 +313,30 @@ fn run_hook(globals: &GlobalArgs, hook: &str, quiet: bool) -> i32 {
267313
return 0;
268314
}
269315

270-
// Validate the payload shape so we don't trigger a full sweep on
271-
// unrelated stdin content. The TS hook ignores payloads missing
272-
// `session_id` / `transcript_path`; mirror that.
273-
match serde_json::from_str::<serde_json::Value>(&raw) {
316+
// Validate the payload shape. The TS hook ignores payloads missing
317+
// `session_id`; mirror that. `transcript_path` is optional — when
318+
// present we drive the single-transcript fast-path, when absent we
319+
// fall back to `ingest_all` so older Claude Code releases that
320+
// elide the field still make forward progress.
321+
let transcript_path = match serde_json::from_str::<serde_json::Value>(&raw) {
274322
Ok(v) => {
275323
let has_session = v.get("session_id").and_then(|x| x.as_str()).is_some();
276-
let has_transcript = v.get("transcript_path").and_then(|x| x.as_str()).is_some();
277-
if !has_session || !has_transcript {
324+
if !has_session {
278325
if !quiet {
279-
eprintln!(
280-
"[burn] ingest: payload missing session_id or transcript_path; ignoring",
281-
);
326+
eprintln!("[burn] ingest: payload missing session_id; ignoring");
282327
}
283328
return 0;
284329
}
330+
v.get("transcript_path")
331+
.and_then(|x| x.as_str())
332+
.map(PathBuf::from)
285333
}
286334
Err(err) => {
287335
eprintln!("[burn] ingest: invalid JSON payload: {err}");
288336
return 0;
289337
}
290-
}
338+
};
291339

292-
// Drive a full sweep. The SDK does not (yet) expose a
293-
// single-transcript verb; `ingest_all` short-circuits unchanged
294-
// cursors so the practical cost is bounded by the new turns this
295-
// hook fires for. Matches the TS hook's "ingest the matching
296-
// session" intent — the Claude transcript that just changed will
297-
// be picked up by `ingest_claude_into` on the same sweep.
298340
let progress = (!quiet).then(|| TaskProgress::new(globals, "ingest"));
299341
if let Some(progress) = &progress {
300342
progress.set_task("opening ledger");
@@ -311,13 +353,16 @@ fn run_hook(globals: &GlobalArgs, hook: &str, quiet: bool) -> i32 {
311353
}
312354
};
313355
if let Some(progress) = &progress {
314-
progress.set_task("scanning sessions");
356+
progress.set_task("ingesting transcript");
315357
}
316358
let opts = match &progress {
317359
Some(progress) => progress.ingest_options(globals.ledger_path.clone()),
318360
None => TaskProgress::quiet_ingest_options(globals.ledger_path.clone()),
319361
};
320-
let result = ingest_all(handle.raw_mut(), &opts);
362+
let result = match transcript_path.as_deref() {
363+
Some(path) => ingest_claude_transcript_path(handle.raw_mut(), path, &opts),
364+
None => ingest_all(handle.raw_mut(), &opts),
365+
};
321366
if let Some(progress) = &progress {
322367
progress.finish_and_clear();
323368
}

0 commit comments

Comments
 (0)