Merge pull request #21 from Factory-AI/ain3sh/showcase-aspect-ratio

Author: factory-ain3sh

plugins/droid-control/remotion/src/compositions/Showcase.tsx

@@ -46,6 +46,13 @@ export const showcaseSchema = z.object({
   clipDuration: z.number().optional(),
   speed: z.number().positive().optional(),
   fidelity: fidelitySchema.optional(),
+  // How clip video is sized inside its panel.
+  // - "contain" (default): preserve aspect ratio, letterbox if needed. Safe default.
+  // - "cover": fill the panel, crop overflow. Use when the clip aspect doesn't match the
+  //   panel aspect (e.g. 16:9 landscape browser capture in a near-square side-by-side panel)
+  //   and you'd rather crop edges than see giant black bars.
+  // - "fill": stretch to fill the panel (distorts aspect). Rarely what you want.
+  objectFit: z.enum(['contain', 'cover', 'fill']).optional(),
 });
 
 const TITLE_DURATION_S = 4;
@@ -68,7 +75,8 @@ const SingleLayout: React.FC<{
   config: ReturnType<typeof getPresetConfig>;
   palette: ReturnType<typeof getPalette>;
   windowTitle?: string;
-}> = ({ clip, config, palette, windowTitle }) => {
+  objectFit: 'contain' | 'cover' | 'fill';
+}> = ({ clip, config, palette, windowTitle, objectFit }) => {
   const { width, height } = useVideoConfig();
 
   const frameW = width - 2 * config.margin;
@@ -91,7 +99,7 @@ const SingleLayout: React.FC<{
       >
         <Video
           src={staticFile(clip)}
-          objectFit="contain"
+          objectFit={objectFit}
           style={{
             width: '100%',
             height: '100%',
@@ -108,7 +116,8 @@ const SideBySideLayout: React.FC<{
   labels: string[];
   config: ReturnType<typeof getPresetConfig>;
   palette: ReturnType<typeof getPalette>;
-}> = ({ clips, labels, config, palette }) => {
+  objectFit: 'contain' | 'cover' | 'fill';
+}> = ({ clips, labels, config, palette, objectFit }) => {
   const { width, height } = useVideoConfig();
 
   const totalW = width - 2 * config.margin;
@@ -139,7 +148,7 @@ const SideBySideLayout: React.FC<{
             >
               <Video
                 src={staticFile(clip)}
-                objectFit="contain"
+                objectFit={objectFit}
                 style={{
                   width: '100%',
                   height: '100%',
@@ -167,6 +176,7 @@ export const ShowcaseComposition: React.FC<z.infer<typeof showcaseSchema>> = (
 
   const titleFrames = TITLE_DURATION_S * fps;
   const clipFrames = Math.ceil((props.clipDuration ?? 60) * fps);
+  const objectFit = props.objectFit ?? 'contain';
 
   const spotlights = useMemo(
     () =>
@@ -220,13 +230,15 @@ export const ShowcaseComposition: React.FC<z.infer<typeof showcaseSchema>> = (
                       labels={props.labels}
                       config={config}
                       palette={palette}
+                      objectFit={objectFit}
                     />
                   ) : props.clips[0] ? (
                     <SingleLayout
                       clip={props.clips[0]}
                       config={config}
                       palette={palette}
                       windowTitle={props.windowTitle}
+                      objectFit={objectFit}
                     />
                   ) : null}
                 </>

plugins/droid-control/skills/capture/SKILL.md

@@ -26,12 +26,29 @@ The command that invoked you should have provided:
 Before recording anything:
 
 - Terminal size is consistent across all sessions (`--cols 120 --rows 36`)
+- **Browser viewport size matches the composition layout** (see "Browser viewport sizing" below) — mismatched aspects letterbox in the final video
 - Branch/worktree paths and env vars are correct
 - Recording format matches the driver: `.cast` for tuistory, `.mp4` for true-input, screenshots for agent-browser
-- If comparing branches, both sessions use identical terminal dimensions and launch parameters
+- If comparing branches, both sessions use identical terminal / viewport dimensions and launch parameters
 - For `droid-dev` captures, `--repo-root` is **mandatory** — `tctl` will refuse to launch without it
 - **Color env vars are set** (see below)
 
+### Browser viewport sizing
+
+Panel aspect ratio in the final composition is **layout-dependent**. At the default 1920×1080 output with factory preset margins, the window-chrome panels that clips render into come out roughly:
+
+| Layout | Panel aspect | Recommended browser viewport |
+|---|---|---|
+| `single` | ~1760×920 (≈16:9 landscape) | **1280×720** or **1440×810** |
+| `side-by-side` | ~872×920 per panel (≈8:9, near-square / slight portrait) | **960×1000**, **900×1000**, or **1024×1080** |
+
+Feeding a 16:9 landscape recording into a near-square side-by-side panel triggers `objectFit: "contain"` letterboxing — you get a thin strip of content with giant black bars above and below. Two ways to avoid it:
+
+1. **Match aspects at capture time** (preferred) — pick the viewport from the table above based on the committed layout.
+2. **Opt into cropping at compose time** — pass `"objectFit": "cover"` in showcase props. Crops the edges of the clip instead of letterboxing. Use when the relevant UI is centered and the clip's edges are expendable.
+
+If you're unsure of the layout when capturing, default to `960×1000` — it is workable in both layouts (slight horizontal letterbox in `single`, no letterbox in `side-by-side`).
+
 ```bash
 TCTL=${DROID_PLUGIN_ROOT}/bin/tctl
 # RUN_ID and RUN_DIR should already be set by the parent (see droid-control ground rule 5)
@@ -62,9 +79,15 @@ $TCTL launch "droid-dev" -s ${RUN_ID}-after --backend tuistory \
   --env FORCE_COLOR=3 --env COLORTERM=truecolor
 ```
 
-**Browser:**
+**Browser:** size the viewport to match the composition layout (see table above).
+
 ```bash
-agent-browser open <url>
+# side-by-side layout → near-square panel
+agent-browser open <url> --viewport 960x1000
+agent-browser record start ${RUN_DIR}/demo.webm
+
+# single layout → 16:9 panel
+agent-browser open <url> --viewport 1280x720
 agent-browser record start ${RUN_DIR}/demo.webm
 ```
 
@@ -126,7 +149,8 @@ Hand these to the **compose** stage:
 - screenshots: [/tmp/proof-1.png, /tmp/proof-2.png]
 - keys: /tmp/keys.tsv (if keystroke logging was requested)
 - driver: tuistory | true-input | agent-browser
-- terminal_size: 120x36
+- terminal_size: 120x36          # for tuistory / true-input
+- viewport: 960x1000             # for agent-browser; report so compose knows the clip aspect
 ```
 
 ## Recovery

plugins/droid-control/skills/compose/SKILL.md

@@ -108,6 +108,26 @@ Set the `speed` prop to hit the target: if the raw recording is 3 minutes and th
 
 `.mp4`, `.webm`, and `.png` clips are passed through to Remotion unchanged except for staging into `public/`. Re-encode non-`.cast` clips manually only if their pixel format or dimensions are invalid.
 
+### Clip aspect ratio (mandatory check for browser captures)
+
+At the default 1920×1080 output with factory preset margins, panels come out roughly:
+
+| Layout | Panel aspect |
+|---|---|
+| `single` | ~1760×920 (≈16:9 landscape) |
+| `side-by-side` | ~872×920 per panel (≈8:9, near-square / slight portrait) |
+
+`.cast` conversions target panel aspect automatically. **Pre-recorded `.mp4` / `.webm` clips do not** — if the clip aspect doesn't match the panel aspect, the clip will letterbox (with the default `objectFit: "contain"`) or crop (with `"cover"`).
+
+**Common pitfall**: browser captures are typically 16:9 landscape (e.g. 1280×720). Dropped into a `side-by-side` layout they render as a thin band with giant black bars above and below.
+
+Two fixes, in priority order:
+
+1. **Re-capture at a panel-friendly viewport** — go back to the capture stage and set viewport to ~960×1000 for `side-by-side`, ~1280×720 for `single`.
+2. **Pass `"objectFit": "cover"` in props** — crops the clip edges to fill the panel. Acceptable when the relevant UI is centered and edges are expendable. Not acceptable if cropped content matters (e.g. sidebar UI cut off).
+
+`.cast` clips rarely need this since their rendered aspect is derived from cols/rows; it's almost always a browser-capture concern.
+
 ### Duration checkpoint (mandatory, before proceeding)
 
 Check whether the planned speed factor produces a final duration within the pacing table's target range:
@@ -182,6 +202,7 @@ Use a run-scoped props path like `$PROPS`; do not reuse a global `/tmp/showcase-
 | `windowTitle` | `string` | no | Text in the window title bar |
 | `width` | `number` | no | Output width (default: 2560 for inspect, else 1920) |
 | `height` | `number` | no | Output height (default: 1440 for inspect, else 1080) |
+| `objectFit` | `"contain" \| "cover" \| "fill"` | no | How each clip fits its panel. Default `"contain"` (letterbox to preserve aspect). Use `"cover"` when clip aspect doesn't match panel aspect and you'd rather crop than see black bars. See "Clip aspect ratio" below. |
 
 ### Preset quick reference