Skip to content

feat(shell): persist background-task output to disk + completion notifications#140

Merged
mabry1985 merged 1 commit into
devfrom
feat/background-shell-disk-capture
Apr 26, 2026
Merged

feat(shell): persist background-task output to disk + completion notifications#140
mabry1985 merged 1 commit into
devfrom
feat/background-shell-disk-capture

Conversation

@mabry1985
Copy link
Copy Markdown

@mabry1985 mabry1985 commented Apr 26, 2026

Why

Background shell tasks lose their output. When the shell tool runs a command with `is_background: true`, it appends `&` and the OS detaches the process. ShellExecutionService's data callback only fires while the wrapper is attached; once it exits, all subsequent stdout/stderr from the detached process flows into the void. Long-running evals (and similar) print results that nothing ever sees, and the agent goes looking for files the runner never wrote.

What

Mirrors cc-2.18's task framework, scoped to local shells:

  • Disk capture from spawn. Background commands now redirect stdout/stderr at the shell level into `//tasks/.output`. The OS keeps writing even after our wrapper exits — the detached process can no longer drop output on the floor.
  • Per-task registry. `BackgroundShellRegistry` tracks every running/completed task with status, exit code, output path, pid.
  • Watcher. Polls a `.exit` sentinel (with PID-liveness fallback) to detect when the bg process actually exits, no matter how detached. Marks the task complete/failed in the registry.
  • Completion notifications. `client.ts` drains pending notifications at the start of each user query and prepends `<task_notification>` blocks (task_id, output_file, status, exit_code, summary) — same pattern used for plan/subagent/arena reminders. Each task is announced exactly once.
  • `bg_stop` tool. SIGTERM the process group, escalate to SIGKILL after 3s, mark the task killed.
  • `/bg` slash command. Lists running and recently-completed background tasks with id, status, age, command, output path, pid.

Files

  • `packages/core/src/backgroundShells/` (new module): `registry.ts`, `diskOutput.ts`, `watcher.ts`, `notifications.ts`, `types.ts`, `index.ts`.
  • `packages/core/src/tools/shell.ts`: rewritten background wrapper to redirect to disk + register task + start watcher; tool result returns the file path and task ID.
  • `packages/core/src/tools/bg-stop.ts` (new): `bg_stop` tool registered alongside the existing `task-*` family.
  • `packages/core/src/core/client.ts`: drains `drainPendingNotifications()` and prepends `<task_notification>` blocks.
  • `packages/core/src/config/config.ts`: `getBackgroundShellRegistry()` lazy accessor.
  • `packages/cli/src/ui/commands/bgCommand.ts` (new): `/bg` slash command.

Tool result before/after

```

before — model has no idea where output went

Background command started. PIDs: 54322 (Use kill to stop)

after — explicit path, model can Read it

Background command started.
Task ID: 7f9c…
Output file: /tmp/proto///tasks/7f9c…output
PID: 54322
Read the output file at any time to check progress. You will
be notified via <task_notification> when the task completes.
Stop early with the bg_stop tool (task_id="7f9c…").
```

Tests

  • 5,337 core + 3,767 cli tests pass.
  • `shell.test.ts` updated for the new wrapper format (4 tests changed) and new mock plumbing for `getBackgroundShellRegistry`/`getSessionId`.

Deferred (follow-up PRs)

  • Auto-background long foreground commands. When a foreground command exceeds an "assistant blocking budget" (~30s), transparently move it to background.
  • Ctrl+B keybinding. User shortcut to background a running foreground task.

Both need foreground shell execution rearchitected so output tees to disk from second 1 and the await can be interrupted mid-flight without killing the process — a bigger blast-radius change worth its own PR.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Added bg slash command to list and monitor long-running background shell tasks, displaying task ID, status, runtime, command, output path, and process ID.
    • Added bg_stop command to gracefully stop running background tasks.
    • Background tasks now automatically generate completion notifications when finished executing.

…fications

Fixes eval-loss bug. Previously, `is_background: true` shell commands
detached via `&` and the OS sent their stdout/stderr to nowhere — protoCLI's
data callback only fires while the wrapper is attached. Long-running
processes like eval suites would print results that nothing ever saw.

Mirrors cc-2.18's task framework, scoped to local shells:

- packages/core/src/backgroundShells/: new module
  * registry.ts — BackgroundShellRegistry: in-memory map of taskId →
    {status, command, outputPath, pid, exitCode, ...}, with
    drainPendingNotifications() for the next-turn injection.
  * diskOutput.ts — path helpers + readBackgroundTaskOutput / Exit / Pid.
    Files live at <projectTempDir>/<sessionId>/tasks/<taskId>.{output,exit,pid}.
  * watcher.ts — polls the .exit sentinel (with PID-liveness fallback),
    marks the task complete/failed in the registry once the bg process
    exits.
  * notifications.ts — builds <task_notification> blocks (task_id,
    output_file, status, exit_code, summary).

- shell.ts: when is_background=true on non-Windows, generate a taskId,
  redirect stdout/stderr at the shell level into the per-task output
  file, capture the bg PID and exit code via sentinel files, register
  the task, and start the watcher. Tool result returns the file path
  + task ID so the model can `Read` it later.

- core/client.ts: drain completed-but-unnotified tasks at the start of
  each user query and prepend <task_notification> blocks to the request,
  matching how plan/subagent/arena reminders are added.

- bg-stop.ts: new bg_stop tool. SIGTERM the process group, escalate to
  SIGKILL after 3s, mark the task killed in the registry. Registered
  as a core tool alongside the existing task-* family.

- bgCommand.ts: new /bg slash command listing running and recent
  background tasks with id, status, age, command, output path, pid.

Tests: shell.test.ts updated for new wrapper format; 55/55 pass. Full
core suite (5,337 tests) and cli suite (3,767 tests) pass.

Deferred (worth follow-up PRs):
- Auto-background of long foreground commands and Ctrl+B keybinding.
  Both need foreground shell execution rearchitected so output tees
  to disk from second 1 and the await can be interrupted mid-flight
  without killing the process.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@mabry1985 mabry1985 merged commit 82b71f0 into dev Apr 26, 2026
@mabry1985 mabry1985 deleted the feat/background-shell-disk-capture branch April 26, 2026 21:19
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 26, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 675ee6d3-0662-459a-92b2-17f2bf1fede5

📥 Commits

Reviewing files that changed from the base of the PR and between 64647ff and 511fdc7.

📒 Files selected for processing (15)
  • packages/cli/src/services/BuiltinCommandLoader.ts
  • packages/cli/src/ui/commands/bgCommand.ts
  • packages/core/src/backgroundShells/diskOutput.ts
  • packages/core/src/backgroundShells/index.ts
  • packages/core/src/backgroundShells/notifications.ts
  • packages/core/src/backgroundShells/registry.ts
  • packages/core/src/backgroundShells/types.ts
  • packages/core/src/backgroundShells/watcher.ts
  • packages/core/src/config/config.ts
  • packages/core/src/core/client.ts
  • packages/core/src/index.ts
  • packages/core/src/tools/bg-stop.ts
  • packages/core/src/tools/shell.test.ts
  • packages/core/src/tools/shell.ts
  • packages/core/src/tools/tool-names.ts

Walkthrough

This PR adds a comprehensive background shell task management system spanning CLI commands, core infrastructure, tools, and integrations. It enables tracking, monitoring, and stopping long-running background tasks with disk-based output capture and user notifications.

Changes

Cohort / File(s) Summary
CLI Command Registration
packages/cli/src/ui/commands/bgCommand.ts, packages/cli/src/services/BuiltinCommandLoader.ts
New bg slash command with list subcommand to display long-running background tasks, showing id, status, runtime, command, output path, and PID. Integrated into built-in command loader.
Background Shell Core Types & Registry
packages/core/src/backgroundShells/types.ts, packages/core/src/backgroundShells/registry.ts
Defines BackgroundShellStatus (running/completed/failed/killed), BackgroundShellTask interface, and BackgroundShellRegistrationInput. Implements BackgroundShellRegistry singleton managing task lifecycle with registration, exit marking, patching, querying, notification draining, and change subscriptions.
Background Shell Disk I/O & Notifications
packages/core/src/backgroundShells/diskOutput.ts, packages/core/src/backgroundShells/notifications.ts, packages/core/src/backgroundShells/watcher.ts
Disk layer for output files and exit/PID sentinels with configurable max-bytes read. Notification builder generating XML task completion blocks. Watcher implementation polling for task completion via sentinel files or POSIX process liveness checks with grace-period retry and weekly timeout.
Background Shell Module Export
packages/core/src/backgroundShells/index.ts
Barrel file re-exporting types, registry, disk I/O, notifications, and watcher modules.
BgStop Tool
packages/core/src/tools/bg-stop.ts, packages/core/src/tools/tool-names.ts
New BG_STOP tool that sends SIGTERM to background task process group with 3-second delayed escalation to SIGKILL. Includes BgStopParams interface and registry lookups. Tool names updated with bg_stop identifier.
Shell Execution with Background Task Integration
packages/core/src/tools/shell.ts, packages/core/src/tools/shell.test.ts
Background execution now wraps commands with disk-backed output capture (.output, .exit, .pid sentinels) for non-Windows. Registers tasks in registry, starts background watcher, and returns task-specific completion messaging. Test fixtures updated for new wrapper behavior and deterministic IDs.
Core Infrastructure Integration
packages/core/src/config/config.ts, packages/core/src/core/client.ts, packages/core/src/index.ts
Config lazily initializes BackgroundShellRegistry and registers BgStopTool in core tools. Client drains pending task notifications during message streaming and converts them to system reminders. Package exports extended to include background shell APIs and bg-stop tool.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Shell as Shell Tool
    participant Registry as Background<br/>Shell Registry
    participant Watcher as Background<br/>Shell Watcher
    participant Disk as Disk Output<br/>Layer
    participant Client as Client
    
    User->>Shell: Execute background command
    Shell->>Disk: Create task dir & output files
    Shell->>Registry: Register task (id, command, outputPath)
    Shell->>Watcher: Start async polling watcher
    Shell-->>User: Task started (id & output path)
    
    Note over Watcher: Poll loop (check sentinel files)
    Watcher->>Disk: Check for .exit sentinel
    Watcher->>Registry: Update task status & exit code
    
    Note over Registry: Task marked completed/failed
    Registry->>Disk: (no action, output persisted)
    
    User->>Client: Send next message
    Client->>Registry: Drain pending notifications
    Registry-->>Client: List of completed tasks
    Client->>Client: Build notification blocks
    Client-->>User: System reminders with task summaries
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/background-shell-disk-capture

Comment @coderabbitai help to get the list of available commands and usage tips.

const commandToExecute = (() => {
if (!shouldRunInBackground || !useDiskCapture) return finalCommand;
// Strip trailing & — we'll re-add it on the subshell wrapper.
const inner = finalCommand.trim().replace(/\s*&\s*$/, '');
mabry1985 added a commit that referenced this pull request Apr 26, 2026
…147)

CodeQL flagged two new alerts on the background-shell wrapper code
landed in #140:
  - js/polynomial-redos at shell.ts:292 ('&+$' on a trimmed string)
  - js/polynomial-redos at shell.ts:305 ('\\s*&\\s*$' on a trimmed string)

Both are low practical risk (the inputs are bounded model-emitted
shell commands) but the alert is blocking the dev → main promotion
in PR #141. Swap each for a plain string-op equivalent — same intent,
no quantifier-on-quantifier shape for the analyzer to flag.

Co-authored-by: Automaker <automaker@localhost>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants