Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
.factory/
node_modules/
2 changes: 1 addition & 1 deletion plugins/droid-control/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ Terminal workflows use `bin/tctl` as the only launch/control boundary. It hides
| `tuistory` | Virtual PTY sessions, deterministic waits/snapshots, asciinema recording at launch. | Fast TUI automation and most demo captures. |
| `true-input` | Headless Wayland compositor, real terminal emulator, native key injection, PTY log/screenshot/video capture. | Real terminal rendering or keyboard-encoding proof. |

`tctl` also enforces Droid CLI launch invariants. `droid-dev` sessions must provide `--repo-root`, which lets `tctl` set `DROID_DEV_REPO_ROOT` and record provenance for the captured branch and commit.
`tctl` also enforces Droid CLI launch invariants. `droid-dev` sessions must provide `--repo-root`, which lets `tctl` set `DROID_DEV_REPO_ROOT` and record provenance for the captured branch and commit. `--cwd` is independent: it sets the child's working directory (recorded in provenance) and defaults to `--repo-root` when unset, so a session can run one worktree's code from a different project directory.

Browser/Electron and native-desktop workflows intentionally do **not** go through `tctl`. They have their own control boundaries: `agent-browser`'s persistent Playwright daemon for DOM snapshots, screenshots, and CDP-connected apps; `cua-driver`'s daemon for accessibility trees and per-`(pid, window_id)` element caches on desktop GUIs.

Expand Down
2 changes: 1 addition & 1 deletion plugins/droid-control/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ For the full rationale and runtime pipeline, see [`ARCHITECTURE.md`](ARCHITECTUR

## Video rendering

The compose stage uses [Remotion](https://www.remotion.dev/) for video compositing. Presets provide window chrome, spacing, palettes, backgrounds, particles, noise, color grading, configurable transitions (`motion-blur`, `flash`, `whip-pan`, `light-leak`, `glitch-lite`), zooms, spotlights, keystroke overlays, section headers, and syntax-highlighted code annotations.
The compose stage uses [Remotion](https://www.remotion.dev/) for video compositing. Presets provide window chrome, spacing, palettes, backgrounds, particles, noise, color grading, configurable transitions (`motion-blur`, `flash`, `whip-pan`, `light-leak`, `glitch-lite`), zooms, spotlights, callout annotations, keystroke overlays, section headers, and syntax-highlighted code annotations.

The `render-showcase.sh` helper owns the full pipeline: `.cast` conversion via `agg`, clip staging, duration detection, Remotion rendering, and cleanup.

Expand Down
34 changes: 21 additions & 13 deletions plugins/droid-control/bin/tctl
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ Launch options:
--terminal <name> Override terminal for true-input
--cols <n> Columns (default: 120)
--rows <n> Rows (default: 36)
--cwd <path> Working directory (tuistory)
--cwd <path> Working directory for the child, both backends
(default: --repo-root when set)
--repo-root <path> Git worktree root for provenance + droid-dev launches
--env <KEY=VALUE> Environment variable (repeatable)
--tmux Wrap the command in tmux with RGB-safe tuistory defaults
Expand Down Expand Up @@ -319,21 +320,25 @@ require_git_worktree_root() {
write_provenance() {
local session="$1"
local repo_root="$2"
local cwd="$3"
local file branch commit
file="$(provenance_file "$session")"

if [[ -z "$repo_root" ]]; then
if [[ -z "$repo_root" && -z "$cwd" ]]; then
rm -f "$file"
return 0
fi

branch="$(git -C "$repo_root" rev-parse --abbrev-ref HEAD 2>/dev/null || printf 'unknown')"
commit="$(git -C "$repo_root" rev-parse HEAD 2>/dev/null || printf 'unknown')"
cat > "$file" <<PROVENANCE
repo_root=$repo_root
branch=$branch
commit=$commit
PROVENANCE
{
if [[ -n "$repo_root" ]]; then
branch="$(git -C "$repo_root" rev-parse --abbrev-ref HEAD 2>/dev/null || printf 'unknown')"
commit="$(git -C "$repo_root" rev-parse HEAD 2>/dev/null || printf 'unknown')"
printf 'repo_root=%s\nbranch=%s\ncommit=%s\n' "$repo_root" "$branch" "$commit"
fi
if [[ -n "$cwd" ]]; then
printf 'cwd=%s\n' "$cwd"
fi
} > "$file"
}

resolve_backend() {
Expand Down Expand Up @@ -937,9 +942,12 @@ cmd_launch() {

if [[ -n "$repo_root" ]]; then
require_git_worktree_root "$repo_root"
if [[ -n "$cwd" && "$cwd" != "$repo_root" ]]; then
die "--cwd must match --repo-root when --repo-root is set"
fi
fi

if [[ -n "$cwd" ]]; then
[[ "$cwd" = /* ]] || die "--cwd must be an absolute path: $cwd"
[[ -d "$cwd" ]] || die "--cwd does not exist: $cwd"
elif [[ -n "$repo_root" ]]; then
cwd="$repo_root"
fi

Expand Down Expand Up @@ -977,7 +985,7 @@ cmd_launch() {
WARMED_UP="0"
REPO_ROOT="$repo_root"
save_session_state "$session"
write_provenance "$session" "$repo_root"
write_provenance "$session" "$repo_root" "$cwd"

if [[ "$BACKEND" == "tuistory" ]]; then
launch_tuistory "$session" "$cols" "$rows" "$record_path"
Expand Down
93 changes: 93 additions & 0 deletions plugins/droid-control/remotion/src/components/CalloutOverlay.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
import { useCurrentFrame, useVideoConfig, interpolate, Easing } from 'remotion';
import type { Effect } from '../lib/schema';
import type { Palette } from '../lib/palettes';

type Callout = Extract<Effect, { fx: 'callout' }>;

const CalloutPill: React.FC<{
callout: Callout;
palette: Palette;
enterFrame: number;
exitFrame: number;
}> = ({ callout, palette, enterFrame, exitFrame }) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();

const isVisible = frame >= enterFrame && frame < exitFrame;
if (!isVisible) return null;

const localFrame = frame - enterFrame;

// Pop-in animation over 0.25s (same overshoot bezier as KeystrokePill)
const enterProgress = interpolate(localFrame, [0, 0.25 * fps], [0, 1], {
easing: Easing.bezier(0.34, 1.56, 0.64, 1),
extrapolateLeft: 'clamp',
extrapolateRight: 'clamp',
});

// Fade out over last 0.2s
const totalFrames = exitFrame - enterFrame;
const fadeOutStart = totalFrames - 0.2 * fps;
const exitOpacity = interpolate(
localFrame,
[fadeOutStart, totalFrames],
[1, 0],
{
extrapolateLeft: 'clamp',
extrapolateRight: 'clamp',
}
);

const scale = interpolate(enterProgress, [0, 1], [0.85, 1]);

return (
<div
style={{
position: 'absolute',
left: callout.at.x,
top: callout.at.y,
transform: `translate(-50%, -50%) scale(${scale})`,
opacity: Math.min(enterProgress, exitOpacity),
backgroundColor: `${palette.surface}E6`,
color: palette.text,
fontSize: 24,
fontFamily: "'Geist', 'Inter', sans-serif",
fontWeight: 500,
lineHeight: 1.4,
padding: '12px 24px',
borderRadius: 12,
border: `1px solid ${palette.border}`,
borderLeft: `3px solid ${palette.accent}`,
backdropFilter: 'blur(12px)',
boxShadow: '0 8px 32px rgba(0, 0, 0, 0.35)',
maxWidth: '40%',
textAlign: 'center',
zIndex: 90,
pointerEvents: 'none',
}}
>
{callout.text}
</div>
);
};

export const CalloutOverlay: React.FC<{
callouts: Callout[];
palette: Palette;
}> = ({ callouts, palette }) => {
const { fps } = useVideoConfig();

return (
<>
{callouts.map((callout) => (
<CalloutPill
key={`${callout.t}-${callout.text}`}
callout={callout}
palette={palette}
enterFrame={Math.round(callout.t * fps)}
exitFrame={Math.round((callout.t + callout.dur) * fps)}
/>
))}
</>
);
};
33 changes: 33 additions & 0 deletions plugins/droid-control/remotion/src/compositions/Showcase.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ import { ZoomEffect } from '../components/ZoomEffect';
import { SectionTransitionOverlay } from '../components/SectionTransition';
import { DroidOutro } from '../components/DroidOutro';
import { CodeAnnotationOverlay } from '../components/CodeAnnotationOverlay';
import { CalloutOverlay } from '../components/CalloutOverlay';

export const showcaseSchema = z.object({
clips: z.array(z.string()),
Expand Down Expand Up @@ -68,6 +69,23 @@ export const showcaseSchema = z.object({
const TITLE_DURATION_S = 4;
const TRANSITION_FRAMES = 15;

// Effect types this composition actually renders. Anything schema-valid but
// absent from this set is a silent no-op — warn instead of dropping quietly.
const RENDERED_FX = new Set(['zoom', 'spotlight', 'callout']);
const warnedUnrenderedFx = new Set<string>();

const warnUnrenderedEffects = (effects: z.infer<typeof effectSchema>[]) => {
for (const effect of effects) {
if (RENDERED_FX.has(effect.fx) || warnedUnrenderedFx.has(effect.fx)) {
continue;
}
warnedUnrenderedFx.add(effect.fx);
console.warn(
`Showcase: effect fx='${effect.fx}' is schema-valid but not rendered by this composition`
);
}
};

const resolveFidelity = (
props: z.infer<typeof showcaseSchema>
): 'compact' | 'standard' | 'inspect' =>
Expand Down Expand Up @@ -211,6 +229,16 @@ export const ShowcaseComposition: React.FC<z.infer<typeof showcaseSchema>> = (
[props.effects]
);

const callouts = useMemo(
() =>
props.effects.filter(
(e): e is Extract<typeof e, { fx: 'callout' }> => e.fx === 'callout'
),
[props.effects]
);

warnUnrenderedEffects(props.effects);

return (
<AbsoluteFill>
<Background palette={palette} config={config} />
Expand Down Expand Up @@ -289,6 +317,11 @@ export const ShowcaseComposition: React.FC<z.infer<typeof showcaseSchema>> = (
/>
))}

{/* Callout annotations: timed text pills at percent positions */}
{callouts.length > 0 && (
<CalloutOverlay callouts={callouts} palette={palette} />
)}

{/* Frosted sweep at section boundaries */}
{props.sections && props.sections.length > 1 && (
<SectionTransitionOverlay sections={props.sections} />
Expand Down
4 changes: 2 additions & 2 deletions plugins/droid-control/skills/compose/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -245,8 +245,8 @@ Use a run-scoped props path like `$PROPS`; do not reuse a global `/tmp/showcase-

| Effect | Props | Description |
|---|---|---|
| `fade-in` | `t`, `dur` | Fade from black |
| `fade-out` | `t`, `dur` | Fade to black |
| `fade-in` | `t`, `dur` | Fade from black (schema-valid but not yet rendered by Showcase — emits a render-time warning) |
| `fade-out` | `t`, `dur` | Fade to black (schema-valid but not yet rendered by Showcase — emits a render-time warning) |
| `zoom` | `t`, `dur`, `to: {x,y,w,h}` | Directed zoom to a target region (30% in, 40% hold, 30% out) |
| `spotlight` | `t`, `dur`, `on: {x,y,w,h}`, `dim?` | Dim everything except a region (`dim`: 0–1, default 0.6) |
| `callout` | `t`, `dur`, `text`, `at: {x,y}` | Text overlay anchored to a point |
Expand Down
1 change: 1 addition & 0 deletions plugins/droid-control/skills/droid-cli/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ $TCTL -s demo press tab
- **No per-branch builds.** One `npm run setup` (in any checkout) installs the shim. Switching branches is instant via `--repo-root`.
- **Prerequisite:** The target worktree must have `node_modules` installed (`npm install` at the repo root). If missing, the bun launch fails.
- **`tctl --repo-root`** sets `DROID_DEV_REPO_ROOT` automatically and pins the session to that worktree.
- **`tctl --cwd`** is independent of `--repo-root`: run droid against one worktree's code while sitting in a different project directory (e.g., a scratch project with its own `.factory/settings.json` hooks). Defaults to `--repo-root` when unset.

### `droid-dev`

Expand Down
38 changes: 38 additions & 0 deletions plugins/droid-control/skills/droid-control/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,44 @@ Terminal drivers use the unified `tctl` wrapper. agent-browser and desktop-contr

Drivers can be combined in one workflow — e.g., `tctl` for a CLI and `agent-browser` for a web UI it interacts with.

## Degraded-tail repro recipe (droid TUI)

Deterministic recipe for reproducing degraded transcript tails in the droid CLI — stranded live tool rows and queued steering messages — without waiting for a slow model turn. The trick: a slow `PreToolUse` hook pins a tool in its executing state for as long as you need.

1. **Scratch project.** Create a throwaway directory (never a real repo — the hook fires on every matching tool call) with a project-local hook that sleeps:

```bash
SCRATCH="$(mktemp -d /tmp/degraded-tail-XXXXXX)"
mkdir -p "$SCRATCH/.factory"
cat > "$SCRATCH/.factory/settings.json" <<'JSON'
{
"hooks": {
"PreToolUse": [
{
"matcher": "TodoWrite",
"hooks": [{ "type": "command", "command": "sleep 120" }]
}
]
}
}
JSON
```

Pick a sleep long enough to interact mid-hook (60–180s) and a matcher for a tool the prompt will reliably trigger (`TodoWrite` fires on any multi-step ask).

2. **Launch with `--cwd` pointed at the scratch project** — `--repo-root` stays on your dev worktree so `droid-dev` provenance still records the code under test:

```bash
$TCTL launch "droid-dev" -s ${RUN_ID}-tail --cwd "$SCRATCH" \
--repo-root /abs/path/to/dev/worktree --record ${RUN_DIR}/tail.cast
```

3. **Trigger the hook**, then degrade the tail while the tool row shows executing:
- **Interrupt mid-hook** (`press escape`) — strands the live tool row: it never resolves to a completed/canceled state in the transcript tail.
- **Steer mid-hook** (`type "..."` + `press enter`) — the steering message queues behind the executing tool instead of interleaving.

**Gotcha:** dev-scope hook settings can silently disable project hooks. If the tool completes instantly, run `/hooks` in the session and check the "Hooks enabled" toggle before debugging the hook config itself.

## Prerequisites

| Stage | Platform | Required | Optional |
Expand Down
2 changes: 1 addition & 1 deletion plugins/droid-control/skills/tuistory/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ $TCTL -s demo close
| `snapshot [--trim]` | Print cleaned text (`--trim` strips trailing blanks) |
| `close` | Tear down session |

Launch options: `--cols <n>`, `--rows <n>`, `--cwd <path>`, `--env KEY=VALUE`, `--record <path>`.
Launch options: `--cols <n>`, `--rows <n>`, `--cwd <path>` (child working directory; defaults to `--repo-root` when set), `--env KEY=VALUE`, `--record <path>`.

## Recording

Expand Down
Loading