feat(pipeline): Director-Critic quality gate before generation

CHANGELOG.md

@@ -2,6 +2,19 @@
 
 All notable changes to OpenReels will be documented in this file.
 
+## [0.15.0] - 2026-04-09
+
+### Added
+- **Director-Critic quality gate**: the pipeline now evaluates and revises the DirectorScore before any expensive generation begins. A Critic agent scores the plan (1-10) and if below 7, provides revision instructions. The Creative Director revises the plan and re-submits. Up to 2 revision rounds, tracking the highest-scoring version. Revision costs ~$0.02-0.13 in LLM calls vs $0.50-2.00+ for regenerating the full pipeline.
+- **Revision cost tracking**: cost estimates now include a "Revise" line item showing actual revision LLM cost. The `CostBreakdown` interface includes `revisionCost` and `revisionRounds` fields.
+- **Worker revision history**: `meta.json` now includes a `revisionHistory` array with per-round critic scores, enabling future UI display of the revision journey.
+
+### Changed
+- The Creative Director's system prompt loading is now a shared `loadDirectorSystemPrompt()` helper, used by both initial generation and revision.
+- The step-6 critic no longer displays "Revision support coming in a future release" since revision is now active.
+- Cost estimation uses the final revised DirectorScore for accurate scene counts when visual types change during revision.
+- Dry-run mode now runs after the revision loop, showing the revised plan instead of the first draft.
+
 ## [0.14.0] - 2026-04-09
 
 ### Added

CLAUDE.md

@@ -46,7 +46,7 @@ fixtures/               # sample DirectorScore JSONs
 ```bash
 pnpm install          # install dependencies
 pnpm start "topic"    # run full pipeline (CLI)
-pnpm test             # run vitest suite (338 tests)
+pnpm test             # run vitest suite (349 tests)
 ```
 
 ### Web UI (Docker Compose)

TODOS.md

@@ -96,6 +96,13 @@
   **Depends on:** TTS alignment layer (Kokoro + Gemini TTS providers)
   Deferred from plan: Unified TTS Alignment Layer (CEO review: user prefers unified voice interface later)
 
+## Critic / Quality Gate
+
+- [ ] **Deterministic structural validation for DirectorScore** — Extract pacing checks (total word count, scene count, per-scene word bounds, consecutive visual_type runs) into a TypeScript validation function. Run it after DirectorScore.parse() in the existing retry loop. Structural violations become retry errors (fed back as error context to the director), not LLM critic concerns. Saves 1-2 LLM calls when the issue is structural, and makes structural quality a guaranteed invariant rather than probabilistic.
+  **Priority:** P3
+  **Depends on:** Director-Critic quality gate (v0.15.0)
+  Deferred from plan: Director-Critic Quality Gate (eng review: outside voice recommended splitting deterministic vs subjective evaluation)
+
 ## Captions
 
 - [ ] **Contextual emphasis on power words** — Creative director marks 1-3 power words per scene with asterisks in script_line. Pipeline strips markers before TTS and records emphasisIndices. CaptionWrapper applies extra animation treatment (larger font-size delta, accent color flash) to emphasis words. Requires: (a) strip asterisks before ALL downstream consumers (critic, image prompter, TTS), not just TTS, (b) creative director prompt update with emphasis marking instructions, (c) emphasisIndices field on CompositionProps (optional, backward compat), (d) CaptionWrapper emphasis detection logic. Per-word asterisks format: `*New* *York*` not `*New York*`.

VERSION

@@ -1 +1 @@
-0.14.0
+0.15.0

docs/content/docs/contributing/architecture.mdx

@@ -39,7 +39,7 @@ AI agent functions that make LLM calls with structured output. Each agent has a
 | `creative-director.ts` | Creative Director | research data + archetype | DirectorScore |
 | `image-prompter.ts` | Image Prompter | raw visual prompt | optimized prompt for image gen |
 | `music-prompter.ts` | Music Prompter | music mood | prompt string for AI music gen |
-| `critic.ts` | Critic | rendered video + score | score 0-10, strengths, weaknesses |
+| `critic.ts` | Critic | DirectorScore + topic | score 0-10, strengths, weaknesses |
 
 ### `src/pipeline/`
 

docs/content/docs/pipeline/critic.mdx

@@ -3,7 +3,7 @@ title: Critic Agent
 description: How the critic evaluates DirectorScore quality using a weighted rubric and flags revision when scores are low.
 ---
 
-The Critic is the final stage of the pipeline. It evaluates the DirectorScore against a structured rubric and produces a quality score. If the score falls below the pass threshold, it identifies the weakest scene and provides specific revision instructions.
+The Critic evaluates the DirectorScore against a structured rubric and produces a quality score. It runs twice in the pipeline: first as a **quality gate** inside the Director stage (before any expensive generation), and again as a **post-mortem** in the final stage. If the score falls below the pass threshold, it identifies the weakest scene and provides specific revision instructions.
 
 ## What it evaluates
 
@@ -73,9 +73,16 @@ interface CritiqueResult {
 }
 ```
 
-## When re-runs happen
+## Quality gate (Director-Critic revision loop)
 
-If `revision_needed` is true and the score is below 7, the Critic logs the weaknesses and notes that revision support is coming in a future release. Currently, the pipeline uses the existing DirectorScore as-is -- the Critic serves as a quality signal rather than triggering automatic re-generation.
+After the Creative Director generates the initial DirectorScore, the Critic evaluates it immediately. If `revision_needed` is true and the score is below 7, the pipeline enters a revision loop:
+
+1. The Critic's `revision_instructions` and `weaknesses` are passed back to the Creative Director
+2. The Director generates a revised DirectorScore addressing the feedback
+3. The Critic re-evaluates the revised plan
+4. The loop runs up to 2 rounds, tracking the highest-scoring revision
+
+This happens before TTS, visuals, or music generation, so revisions cost only LLM calls (~$0.02-0.13) instead of re-running the full pipeline ($0.50-2.00+). If the Critic fails during evaluation, the loop exits gracefully and proceeds with the current best score.
 
 The critique results are emitted through the progress callback so the web UI can display the score, strengths, and weaknesses to the user.
 

docs/content/docs/pipeline/overview.mdx

@@ -14,8 +14,9 @@ Topic
 Research ---- web search grounds the script in facts
   |
   v
-Director ---- writes the DirectorScore (scenes, archetype, music mood)
-  |
+Director ---- writes the DirectorScore, then Critic evaluates it
+  |    ^            (revision loop: up to 2 rounds until score >= 7)
+  |    |
   v
 TTS --------- generates voiceover audio with word-level timestamps
   |
@@ -89,7 +90,7 @@ After all stages complete, actual LLM token usage is aggregated and the real cos
 
 The pipeline supports two early exit paths:
 
-- **Dry run** (`--dry-run`) -- stops after the Director stage and prints the DirectorScore JSON. No assets are generated.
+- **Dry run** (`--dry-run`) -- stops after the Director stage (including the quality gate revision loop) and prints the final DirectorScore JSON. No assets are generated.
 - **Cost rejection** -- if the user declines at the cost estimate prompt, the pipeline exits cleanly.
 
 Both paths still write `log.json` to the output directory.

src/agents/creative-director.test.ts

@@ -1,5 +1,9 @@
-import { describe, expect, it } from "vitest";
-import { buildPacingInstruction, PACING_CONFIG } from "./creative-director.js";
+import { describe, expect, it, vi } from "vitest";
+import { buildPacingInstruction, PACING_CONFIG, reviseDirectorScore } from "./creative-director.js";
+import type { DirectorScore } from "../schema/director-score.js";
+import type { LLMProvider } from "../schema/providers.js";
+import type { CritiqueResult } from "./critic.js";
+import type { ResearchResult } from "./research.js";
 
 describe("buildPacingInstruction", () => {
   // Path 2: explicit archetype — derive tier from config
@@ -102,3 +106,81 @@ describe("PACING_CONFIG", () => {
     expect(PACING_CONFIG.cinematic.max).toBeLessThan(PACING_CONFIG.fast.max);
   });
 });
+
+// ── reviseDirectorScore tests ────────────────────────────────────────────────
+
+const baseScore: DirectorScore = {
+  emotional_arc: "curiosity-to-wisdom",
+  archetype: "infographic",
+  music_mood: "epic_cinematic",
+  scenes: [
+    { visual_type: "text_card", visual_prompt: "Title", motion: "static", script_line: "Did you know this?", transition: null },
+    { visual_type: "ai_image", visual_prompt: "A diagram", motion: "zoom_in", script_line: "Here is the truth.", transition: "crossfade" },
+    { visual_type: "stock_video", visual_prompt: "Ocean", motion: "static", script_line: "It changed everything.", transition: "slide_left" },
+    { visual_type: "ai_image", visual_prompt: "Chart", motion: "pan_right", script_line: "The numbers prove it.", transition: "crossfade" },
+    { visual_type: "stock_image", visual_prompt: "Sunset", motion: "zoom_out", script_line: "Think about that.", transition: "crossfade" },
+    { visual_type: "text_card", visual_prompt: "CTA", motion: "static", script_line: "What do you think?", transition: "wipe" },
+    { visual_type: "ai_image", visual_prompt: "Finale", motion: "zoom_in", script_line: "Comment below!", transition: null },
+    { visual_type: "stock_video", visual_prompt: "Stars", motion: "static", script_line: "Follow for more.", transition: "crossfade" },
+  ],
+};
+
+const baseResearch: ResearchResult = {
+  summary: "Test topic summary",
+  key_facts: ["fact1", "fact2"],
+  mood: "informative",
+  sources: [],
+};
+
+const baseCritique: CritiqueResult = {
+  score: 5,
+  strengths: ["good hook"],
+  weaknesses: ["weak pacing", "repetitive visuals"],
+  revision_needed: true,
+  revision_instructions: "Improve pacing in scenes 3-5 and add more visual variety.",
+  weakest_scene_index: 3,
+};
+
+function mockRevisionLLM(): LLMProvider & { lastUserMessage: string } {
+  const mock = {
+    id: "anthropic" as const,
+    lastUserMessage: "",
+    generate: vi.fn(async ({ userMessage }: { userMessage: string }) => {
+      mock.lastUserMessage = userMessage;
+      return {
+        data: { ...baseScore },
+        usage: { inputTokens: 200, outputTokens: 100 },
+      };
+    }),
+  };
+  return mock as unknown as LLMProvider & { lastUserMessage: string };
+}
+
+describe("reviseDirectorScore", () => {
+  it("generates a revised DirectorScore from critique feedback", async () => {
+    const llm = mockRevisionLLM();
+    const result = await reviseDirectorScore(llm, "test topic", baseResearch, baseScore, baseCritique);
+    expect(result.data.archetype).toBe("infographic");
+    expect(result.data.scenes.length).toBeGreaterThanOrEqual(3);
+    expect(result.usage.inputTokens).toBe(200);
+  });
+
+  it("includes critique details in the user message", async () => {
+    const llm = mockRevisionLLM();
+    await reviseDirectorScore(llm, "test topic", baseResearch, baseScore, baseCritique);
+    expect(llm.lastUserMessage).toContain("score: 5/10");
+    expect(llm.lastUserMessage).toContain("weak pacing");
+    expect(llm.lastUserMessage).toContain("Improve pacing in scenes 3-5");
+    expect(llm.lastUserMessage).toContain("Weakest scene: Scene 3");
+  });
+
+  it("falls back to weaknesses when revision_instructions is null", async () => {
+    const llm = mockRevisionLLM();
+    const critique: CritiqueResult = {
+      ...baseCritique,
+      revision_instructions: null,
+    };
+    await reviseDirectorScore(llm, "test topic", baseResearch, baseScore, critique);
+    expect(llm.lastUserMessage).toContain("weak pacing; repetitive visuals");
+  });
+});

src/agents/creative-director.ts

@@ -7,6 +7,7 @@ import { loadPlaybook } from "../config/playbook.js";
 import { DirectorScore, Motion, MusicMood, TransitionType, VisualType } from "../schema/director-score.js";
 import type { LLMProvider, LLMUsage } from "../schema/providers.js";
 import type { ResearchResult } from "./research.js";
+import type { CritiqueResult } from "./critic.js";
 
 const SYSTEM_PROMPT_PATH = path.join(process.cwd(), "prompts", "creative-director.md");
 
@@ -34,12 +35,8 @@ export interface DirectorScoreOutput {
   usage: LLMUsage;
 }
 
-export async function generateDirectorScore(
-  llm: LLMProvider,
-  topic: string,
-  researchContext: ResearchResult,
-  options?: { archetype?: string; pacing?: string; videoEnabled?: boolean },
-): Promise<DirectorScoreOutput> {
+/** Load the creative director system prompt with playbook injection */
+function loadDirectorSystemPrompt(): string {
   let systemPrompt = buildDefaultPrompt();
 
   try {
@@ -56,6 +53,17 @@ export async function generateDirectorScore(
     console.warn(`[creative-director] Playbook not loaded: ${err}`);
   }
 
+  return systemPrompt;
+}
+
+export async function generateDirectorScore(
+  llm: LLMProvider,
+  topic: string,
+  researchContext: ResearchResult,
+  options?: { archetype?: string; pacing?: string; videoEnabled?: boolean },
+): Promise<DirectorScoreOutput> {
+  const systemPrompt = loadDirectorSystemPrompt();
+
   const archetypes = listArchetypes();
   const archetypeInstruction = options?.archetype
     ? `Use the "${options.archetype}" archetype.`
@@ -180,3 +188,92 @@ Per-scene word budget: ${cfg.wordsPerScene} words. Total word budget: ${cfg.tota
 }
 
 export { PACING_CONFIG };
+
+// ── Revision ─────────────────────────────────────────────────────────────────
+
+export async function reviseDirectorScore(
+  llm: LLMProvider,
+  topic: string,
+  researchContext: ResearchResult,
+  originalScore: DirectorScore,
+  critique: CritiqueResult,
+  options?: { archetype?: string; pacing?: string; videoEnabled?: boolean },
+): Promise<DirectorScoreOutput> {
+  const systemPrompt = loadDirectorSystemPrompt();
+
+  // Build revision instructions from critique, guarding nullable revision_instructions
+  const revisionGuidance = critique.revision_instructions
+    ?? `Address these weaknesses: ${critique.weaknesses.join("; ")}`;
+
+  const pacingInstruction = buildPacingInstruction(options?.archetype, options?.pacing);
+
+  const videoEnabled = options?.videoEnabled ?? false;
+  const visualTypes = videoEnabled
+    ? "all 5 visual types (ai_image, ai_video, stock_image, stock_video, text_card)"
+    : "all 4 visual types (ai_image, stock_image, stock_video, text_card)";
+
+  const userMessage = `Topic: ${topic}
+
+Research context:
+${researchContext.summary}
+
+Key facts:
+${researchContext.key_facts.map((f) => `- ${f}`).join("\n")}
+
+Mood: ${researchContext.mood}
+
+${pacingInstruction}
+Use ${visualTypes}.
+
+## Current Plan (score: ${critique.score}/10)
+
+${JSON.stringify(originalScore, null, 2)}
+
+## Critic Feedback
+
+Strengths: ${critique.strengths.join(", ")}
+Weaknesses: ${critique.weaknesses.join(", ")}
+${critique.weakest_scene_index != null ? `Weakest scene: Scene ${critique.weakest_scene_index}` : ""}
+
+## Revision Instructions
+
+${revisionGuidance}
+
+Revise the DirectorScore to address the weaknesses while preserving the strengths.
+Keep the same archetype. Maintain the GOLDEN RULE: never use the same visual_type more than 2 times in a row.`;
+
+  const maxRetries = 2;
+  let lastError: Error | null = null;
+  const totalUsage: LLMUsage = { inputTokens: 0, outputTokens: 0 };
+
+  for (let attempt = 0; attempt < maxRetries; attempt++) {
+    try {
+      const result = await llm.generate({
+        systemPrompt,
+        userMessage:
+          attempt > 0
+            ? `${userMessage}\n\nPREVIOUS ATTEMPT FAILED: ${lastError?.message}. Fix the issue.`
+            : userMessage,
+        schema: DirectorScoreRaw,
+      });
+
+      totalUsage.inputTokens += result.usage.inputTokens;
+      totalUsage.outputTokens += result.usage.outputTokens;
+
+      const validated = DirectorScore.parse(result.data);
+
+      // Prevent archetype drift: the LLM may change the archetype during revision
+      // despite prompt instructions. Force it back to the original.
+      if (validated.archetype !== originalScore.archetype) {
+        (validated as { archetype: string }).archetype = originalScore.archetype;
+      }
+
+      return { data: validated, usage: totalUsage };
+    } catch (err) {
+      lastError = err instanceof Error ? err : new Error(String(err));
+      console.warn(`[creative-director] Revision attempt ${attempt + 1} failed: ${lastError.message}`);
+    }
+  }
+
+  throw new Error(`Revision failed after ${maxRetries} attempts: ${lastError?.message}`);
+}

src/cli/cost-estimator.test.ts

@@ -70,7 +70,7 @@ describe("estimateCost", () => {
     ]);
     const result = estimateCost(score);
     expect(result.totalCost).toBeCloseTo(
-      result.llmCost + result.ttsCost + result.imageCost + result.videoCost + result.musicCost,
+      result.llmCost + result.revisionCost + result.ttsCost + result.imageCost + result.videoCost + result.musicCost,
     );
   });
 
@@ -119,6 +119,30 @@ describe("estimateCost", () => {
     expect(fal.videoCost).toBeGreaterThan(gemini.videoCost);
   });
 
+  it("includes revision cost with separate evaluation and revision counts", () => {
+    const score = makeScore([{ visual_type: "ai_image", script_line: "Test" }]);
+    // No gate activity
+    const noGate = estimateCost(score, "gemini", "elevenlabs", undefined, "anthropic", "bundled", 0, 0);
+    expect(noGate.revisionCost).toBe(0);
+    expect(noGate.details.gateEvaluations).toBe(0);
+    expect(noGate.details.revisionRounds).toBe(0);
+
+    // Gate only (score >= 7 on first try): 1 eval, 0 revisions
+    const gateOnly = estimateCost(score, "gemini", "elevenlabs", undefined, "anthropic", "bundled", 1, 0);
+    expect(gateOnly.revisionCost).toBeGreaterThan(0);
+    expect(gateOnly.details.gateEvaluations).toBe(1);
+    expect(gateOnly.details.revisionRounds).toBe(0);
+
+    // Full revision: 2 evals + 1 revision
+    const fullRevision = estimateCost(score, "gemini", "elevenlabs", undefined, "anthropic", "bundled", 2, 1);
+    expect(fullRevision.revisionCost).toBeGreaterThan(gateOnly.revisionCost);
+    expect(fullRevision.details.gateEvaluations).toBe(2);
+    expect(fullRevision.details.revisionRounds).toBe(1);
+
+    // Gate-only should cost less than a full revision (no director call)
+    expect(gateOnly.revisionCost).toBeLessThan(fullRevision.revisionCost);
+  });
+
   it("uses gemini LLM pricing when llmProvider is gemini", () => {
     const score = makeScore([{ visual_type: "ai_image", script_line: "Test" }]);
     const anthropic = estimateCost(score, "gemini", "elevenlabs", undefined, "anthropic");