[Phase 4.B] Worktree isolation for parallel batch execution

[frankbria]

Overview

Each parallel task worker gets its own git worktree, eliminating filesystem conflicts during cf work batch run --strategy parallel.

Motivation

The current parallel strategy runs all workers on the same filesystem. This causes:

• Git index lock conflicts when multiple agents commit simultaneously
• Agents overwriting each other's uncommitted changes
• Non-deterministic outcomes in parallel batch runs

Deliverables

codeframe/core/sandbox/worktree.py

Worktree lifecycle management:

class WorktreeContext:
    def create(task_id: str, base_path: Path) -> WorktreeContext
    def cleanup(self) -> None  # removes worktree, merges or discards
    @property
    def path(self) -> Path

Integration with conductor.py

• When IsolationLevel.WORKTREE, each parallel worker calls WorktreeContext.create()
• Worker receives context.workspace_path pointing to its worktree
• On completion, changes are merged back (or blocker raised on conflict)
• cleanup() removes the worktree regardless of outcome

Atomic registration

• Prevent race conditions when multiple workers create worktrees simultaneously
• File-lock or DB-based registration in .codeframe/worktrees.json

Auto-cleanup

• Orphaned worktrees (from crashed workers) cleaned up on next cf work batch run
• cf env doctor reports stale worktrees

Acceptance Criteria

• WorktreeContext in core/sandbox/worktree.py
• conductor.py uses worktrees when --isolation worktree
• Each parallel worker operates on isolated worktree.path
• Successful task merges changes back to base branch
• Failed task discards worktree without affecting base
• Atomic registration prevents race conditions
• Auto-cleanup on next run or cf env doctor
• Integration tests: parallel tasks on same files no longer conflict

Dependencies

• Depends on: [Phase 4.B] ExecutionContext abstraction: local | worktree | e2b-sandbox #532 (ExecutionContext abstraction)

Track

Phase 4.B — Execution Environment Layer

[traycerai]

Warning

Ticket Assist will be discontinued on May 1, 2026. After that date, the feature will no longer be available. We suggest installing the Traycer IDE Extension for continued AI-powered, spec-driven development. Thank you for choosing Traycer!

Plan

Observations

The codebase already has a solid foundation for this feature:

• file:codeframe/core/worktrees.py — TaskWorktree class with create(), merge_back(), and cleanup() is fully implemented.
• file:codeframe/core/conductor.py — _execute_task_subprocess() already accepts a worktree_path: Optional[Path] parameter and uses it as cwd, and BatchRun already has an isolate: bool = True field.
• file:codeframe/cli/app.py — batch_run command does not yet pass isolate to conductor.start_batch(), and there is no --isolation CLI flag.
• file:codeframe/cli/env_commands.py — doctor command exists but has no worktree awareness.
• No atomic registration file (.codeframe/worktrees.json) or orphan-cleanup logic exists yet.

Approach

Wire the existing TaskWorktree into _execute_group_parallel() and _execute_single_task() in conductor.py, add atomic registration via a JSON file protected by a threading lock, add orphan cleanup at batch-start and in cf env doctor, and expose --isolation worktree on the CLI. This avoids introducing new libraries and stays consistent with the existing subprocess + threading model.

---

Implementation Steps

1. Add WorktreeRegistry to codeframe/core/worktrees.py

Add a new class WorktreeRegistry below TaskWorktree that manages .codeframe/worktrees.json atomically:

• register(workspace_path, task_id, batch_id) — acquires a threading.Lock (module-level), reads the JSON file (creating it if absent), appends {task_id, batch_id, created_at, pid}, writes back atomically (write to a .tmp file then os.replace).
• unregister(workspace_path, task_id) — same lock, removes the entry for task_id.
• list_stale(workspace_path) — reads the registry and returns entries whose pid is no longer alive (os.kill(pid, 0) raises ProcessLookupError).
• cleanup_stale(workspace_path) — calls list_stale(), then for each stale entry calls TaskWorktree().cleanup(workspace_path, task_id) and unregister().

Use a module-level threading.Lock (already consistent with how _batch_db_lock is used in conductor.py). No new library is needed.

---

2. Add get_base_branch() helper to codeframe/core/worktrees.py

Add a small module-level function get_base_branch(workspace_path: Path) -> str that runs git rev-parse --abbrev-ref HEAD and returns the result (defaulting to "main"). This is needed by the conductor so it doesn't hard-code the branch name.

---

3. Wire worktree isolation into conductor.py

3a. Update _execute_single_task()

Add a worktree: Optional[TaskWorktree] = None parameter. When worktree is provided:

1. Call worktree.create(workspace.repo_path, task_id, base_branch) to get worktree_path.
2. Register via WorktreeRegistry.register(workspace.repo_path, task_id, batch.id).
3. Pass worktree_path to _execute_task_subprocess() (the parameter already exists).
4. On completion: if result_status == COMPLETED, call worktree.merge_back(); if merge fails, set result_status to FAILED and log the conflict details.
5. Always call worktree.cleanup() and WorktreeRegistry.unregister() in a finally block.

3b. Update _execute_group_parallel()

Add a worktree: Optional[TaskWorktree] = None parameter. Inside the execute_task() closure, apply the same create → register → execute → merge/discard → cleanup → unregister pattern per task. Each task gets its own independent worktree path, so there is no shared state between threads.

3c. Update _execute_parallel() and _execute_serial()

At the top of each function, determine whether to use worktrees:

use_worktree = batch.isolate and batch.strategy == "parallel"
worktree = TaskWorktree() if use_worktree else None

Pass worktree down to _execute_single_task() and _execute_group_parallel().

Serial execution does not need worktree isolation (no concurrent filesystem access), so worktree remains None for serial runs.

3d. Add orphan cleanup at batch start

At the beginning of _execute_parallel(), before creating the execution plan, call WorktreeRegistry.cleanup_stale(workspace.repo_path) and log any cleaned-up entries.

---

4. Add --isolation flag to batch_run CLI in codeframe/cli/app.py

Add a new Typer option to batch_run():

isolation: str = typer.Option(
    "worktree",
    "--isolation",
    help="Isolation mode for parallel workers: 'worktree' (default) or 'none'",
    click_type=click.Choice(["worktree", "none"], case_sensitive=False),
)

Map isolation == "worktree" → isolate=True, isolation == "none" → isolate=False, and pass isolate to conductor.start_batch(). Update the "Batch Execution Plan" console output to show the isolation mode.

---

5. Add stale worktree reporting to cf env doctor

In file:codeframe/cli/env_commands.py, inside the doctor() command, after the existing tools table:

1. Import WorktreeRegistry from codeframe.core.worktrees.
2. Call WorktreeRegistry().list_stale(project_path).
3. If any stale entries exist, print a warning section listing each stale task_id and its pid, and suggest running cf work batch run (which auto-cleans) or manually removing .codeframe/worktrees/.

---

6. Add list_worktrees() to codeframe/core/worktrees.py

A simple function list_worktrees(workspace_path: Path) -> list[dict] that reads .codeframe/worktrees.json and returns the parsed list (empty list if file absent). Used by cf env doctor and any future tooling.

---

Data Flow for a Parallel Batch with Worktree Isolation

sequenceDiagram
    participant CLI as cf work batch run
    participant Conductor as conductor._execute_parallel()
    participant Registry as WorktreeRegistry
    participant WT as TaskWorktree
    participant Sub as subprocess (cf work start)
    participant Git as git

    CLI->>Conductor: start_batch(isolate=True, strategy="parallel")
    Conductor->>Registry: cleanup_stale(workspace_path)
    loop For each parallel group
        Conductor->>WT: create(workspace_path, task_id, base_branch)
        WT->>Git: git worktree add .codeframe/worktrees/<task_id> -b cf/<task_id>
        Conductor->>Registry: register(workspace_path, task_id, batch_id, pid)
        Conductor->>Sub: Popen(cwd=worktree_path)
        Sub-->>Conductor: exit (COMPLETED / FAILED)
        alt COMPLETED
            Conductor->>WT: merge_back(workspace_path, task_id, base_branch)
            Git-->>Conductor: MergeResult(success=True/False)
        end
        Conductor->>WT: cleanup(workspace_path, task_id)
        Conductor->>Registry: unregister(workspace_path, task_id)
    end
    Conductor-->>CLI: BatchRun with results

Loading

---

Summary of Changes

File	Change
file:codeframe/core/worktrees.py	Add WorktreeRegistry (register/unregister/list_stale/cleanup_stale), get_base_branch(), list_worktrees()
file:codeframe/core/conductor.py	Wire TaskWorktree + WorktreeRegistry into _execute_single_task(), _execute_group_parallel(), _execute_parallel(); add orphan cleanup at batch start
file:codeframe/cli/app.py	Add --isolation option to batch_run; pass isolate to conductor.start_batch()
file:codeframe/cli/env_commands.py	Add stale worktree reporting section to doctor() command

Import In IDE

🤖 Prompt for AI Agents

## Observations

The codebase already has a solid foundation for this feature:
- `file:codeframe/core/worktrees.py` — `TaskWorktree` class with `create()`, `merge_back()`, and `cleanup()` is fully implemented.
- `file:codeframe/core/conductor.py` — `_execute_task_subprocess()` already accepts a `worktree_path: Optional[Path]` parameter and uses it as `cwd`, and `BatchRun` already has an `isolate: bool = True` field.
- `file:codeframe/cli/app.py` — `batch_run` command does **not** yet pass `isolate` to `conductor.start_batch()`, and there is no `--isolation` CLI flag.
- `file:codeframe/cli/env_commands.py` — `doctor` command exists but has no worktree awareness.
- No atomic registration file (`.codeframe/worktrees.json`) or orphan-cleanup logic exists yet.

## Approach

Wire the existing `TaskWorktree` into `_execute_group_parallel()` and `_execute_single_task()` in `conductor.py`, add atomic registration via a JSON file protected by a threading lock, add orphan cleanup at batch-start and in `cf env doctor`, and expose `--isolation worktree` on the CLI. This avoids introducing new libraries and stays consistent with the existing subprocess + threading model.

---

## Implementation Steps

### 1. Add `WorktreeRegistry` to `codeframe/core/worktrees.py`

Add a new class `WorktreeRegistry` below `TaskWorktree` that manages `.codeframe/worktrees.json` atomically:

- **`register(workspace_path, task_id, batch_id)`** — acquires a `threading.Lock` (module-level), reads the JSON file (creating it if absent), appends `{task_id, batch_id, created_at, pid}`, writes back atomically (write to a `.tmp` file then `os.replace`).
- **`unregister(workspace_path, task_id)`** — same lock, removes the entry for `task_id`.
- **`list_stale(workspace_path)`** — reads the registry and returns entries whose `pid` is no longer alive (`os.kill(pid, 0)` raises `ProcessLookupError`).
- **`cleanup_stale(workspace_path)`** — calls `list_stale()`, then for each stale entry calls `TaskWorktree().cleanup(workspace_path, task_id)` and `unregister()`.

Use a module-level `threading.Lock` (already consistent with how `_batch_db_lock` is used in `conductor.py`). No new library is needed.

---

### 2. Add `get_base_branch()` helper to `codeframe/core/worktrees.py`

Add a small module-level function `get_base_branch(workspace_path: Path) -> str` that runs `git rev-parse --abbrev-ref HEAD` and returns the result (defaulting to `"main"`). This is needed by the conductor so it doesn't hard-code the branch name.

---

### 3. Wire worktree isolation into `conductor.py`

#### 3a. Update `_execute_single_task()`

Add a `worktree: Optional[TaskWorktree] = None` parameter. When `worktree` is provided:
1. Call `worktree.create(workspace.repo_path, task_id, base_branch)` to get `worktree_path`.
2. Register via `WorktreeRegistry.register(workspace.repo_path, task_id, batch.id)`.
3. Pass `worktree_path` to `_execute_task_subprocess()` (the parameter already exists).
4. On completion: if `result_status == COMPLETED`, call `worktree.merge_back()`; if merge fails, set `result_status` to `FAILED` and log the conflict details.
5. Always call `worktree.cleanup()` and `WorktreeRegistry.unregister()` in a `finally` block.

#### 3b. Update `_execute_group_parallel()`

Add a `worktree: Optional[TaskWorktree] = None` parameter. Inside the `execute_task()` closure, apply the same create → register → execute → merge/discard → cleanup → unregister pattern per task. Each task gets its own independent worktree path, so there is no shared state between threads.

#### 3c. Update `_execute_parallel()` and `_execute_serial()`

At the top of each function, determine whether to use worktrees:
```
use_worktree = batch.isolate and batch.strategy == "parallel"
worktree = TaskWorktree() if use_worktree else None
```
Pass `worktree` down to `_execute_single_task()` and `_execute_group_parallel()`.

> Serial execution does not need worktree isolation (no concurrent filesystem access), so `worktree` remains `None` for serial runs.

#### 3d. Add orphan cleanup at batch start

At the beginning of `_execute_parallel()`, before creating the execution plan, call `WorktreeRegistry.cleanup_stale(workspace.repo_path)` and log any cleaned-up entries.

---

### 4. Add `--isolation` flag to `batch_run` CLI in `codeframe/cli/app.py`

Add a new Typer option to `batch_run()`:

```
isolation: str = typer.Option(
    "worktree",
    "--isolation",
    help="Isolation mode for parallel workers: 'worktree' (default) or 'none'",
    click_type=click.Choice(["worktree", "none"], case_sensitive=False),
)
```

Map `isolation == "worktree"` → `isolate=True`, `isolation == "none"` → `isolate=False`, and pass `isolate` to `conductor.start_batch()`. Update the "Batch Execution Plan" console output to show the isolation mode.

---

### 5. Add stale worktree reporting to `cf env doctor`

In `file:codeframe/cli/env_commands.py`, inside the `doctor()` command, after the existing tools table:

1. Import `WorktreeRegistry` from `codeframe.core.worktrees`.
2. Call `WorktreeRegistry().list_stale(project_path)`.
3. If any stale entries exist, print a warning section listing each stale `task_id` and its `pid`, and suggest running `cf work batch run` (which auto-cleans) or manually removing `.codeframe/worktrees/`.

---

### 6. Add `list_worktrees()` to `codeframe/core/worktrees.py`

A simple function `list_worktrees(workspace_path: Path) -> list[dict]` that reads `.codeframe/worktrees.json` and returns the parsed list (empty list if file absent). Used by `cf env doctor` and any future tooling.

---

## Data Flow for a Parallel Batch with Worktree Isolation

```mermaid
sequenceDiagram
    participant CLI as cf work batch run
    participant Conductor as conductor._execute_parallel()
    participant Registry as WorktreeRegistry
    participant WT as TaskWorktree
    participant Sub as subprocess (cf work start)
    participant Git as git

    CLI->>Conductor: start_batch(isolate=True, strategy="parallel")
    Conductor->>Registry: cleanup_stale(workspace_path)
    loop For each parallel group
        Conductor->>WT: create(workspace_path, task_id, base_branch)
        WT->>Git: git worktree add .codeframe/worktrees/<task_id> -b cf/<task_id>
        Conductor->>Registry: register(workspace_path, task_id, batch_id, pid)
        Conductor->>Sub: Popen(cwd=worktree_path)
        Sub-->>Conductor: exit (COMPLETED / FAILED)
        alt COMPLETED
            Conductor->>WT: merge_back(workspace_path, task_id, base_branch)
            Git-->>Conductor: MergeResult(success=True/False)
        end
        Conductor->>WT: cleanup(workspace_path, task_id)
        Conductor->>Registry: unregister(workspace_path, task_id)
    end
    Conductor-->>CLI: BatchRun with results
```

---

## Summary of Changes

| File | Change |
|---|---|
| `file:codeframe/core/worktrees.py` | Add `WorktreeRegistry` (register/unregister/list_stale/cleanup_stale), `get_base_branch()`, `list_worktrees()` |
| `file:codeframe/core/conductor.py` | Wire `TaskWorktree` + `WorktreeRegistry` into `_execute_single_task()`, `_execute_group_parallel()`, `_execute_parallel()`; add orphan cleanup at batch start |
| `file:codeframe/cli/app.py` | Add `--isolation` option to `batch_run`; pass `isolate` to `conductor.start_batch()` |
| `file:codeframe/cli/env_commands.py` | Add stale worktree reporting section to `doctor()` command |

---

Execution Information

Branch: main
Commit: 7d5caab

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coderabbitai]

⚠️ Possible Duplicate Issue(s)

• [Phase 4] Worktree-Per-Task Isolation for Parallel Batch Execution #418

🔗 Related PRs

#436 - feat(adapters): detect modified files via git diff after subprocess execution [merged]
#439 - feat(core): continuous reconciliation during batch execution [merged]
#441 - feat(core): worktree-per-task isolation for parallel batch execution [merged]
#450 - feat(config): dynamic configuration reload during batch execution (#402) [merged]
#514 - docs(roadmap): restructure Phase 4 with execution environment layer [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!