[codex] remove legacy prompt optimizer surface (#20)

Author: drewstone

CHANGELOG.md

@@ -1,5 +1,23 @@
 # Changelog
 
+## 0.19.0 — legacy optimizer removal
+
+### Removed
+
+- Removed the legacy pairwise prompt optimizer surface:
+  `PromptOptimizer`, `OptimizationLoop`, and their associated root-exported
+  types are gone. The blessed optimization path is now
+  `runMultiShotOptimization` for task trajectories and the steering-specific
+  optimizers for explicit steering tables.
+- Removed the old `PromptVariant` root export. Public callers should use
+  `MultiShotVariant` for multi-shot trajectory optimization or
+  `EvolvableVariant` for the lower-level prompt/code evolution core.
+
+### Changed
+
+- Documentation now points optimization users at `runMultiShotOptimization`
+  instead of the removed pairwise prompt optimizer.
+
 ## 0.18.0 — multi-shot optimization
 
 ### Added

README.md

@@ -81,7 +81,7 @@ The recipe for a code-generator eval is in [`SKILL.md` §Minimal working path](.
 | `runAgentControlLoop` | Policy-based runtime for agentic tasks: observe typed state, validate, decide, act, repeat with budgets, tracing, and stuck-loop guards. | [control-runtime.md](./docs/control-runtime.md) |
 | `FeedbackTrajectory`, `InMemoryFeedbackTrajectoryStore`, `FileSystemFeedbackTrajectoryStore` | Human/environment feedback loops: capture approvals, rejections, choices, revisions, metrics, and policy blocks as train/dev/test/holdout examples. | [feedback-trajectories.md](./docs/feedback-trajectories.md) |
 | `evaluateActionPolicy` | Generic action preflight for approval, budget, expected-outcome, and kill-criteria checks. | [feature-guide.md](./docs/feature-guide.md) |
-| `ExperimentTracker`, `PromptOptimizer`, `bisector` | A/B prompts, optimize steering, bisect regressions. | SKILL.md |
+| `ExperimentTracker`, steering optimizers, `bisector` | A/B prompts, optimize steering, bisect regressions. | SKILL.md |
 | `runMultiShotOptimization`, `trialTraceFromMultiShotTrial` | GEPA-style optimization for variable-length agent trajectories with ASI, paired seeds, and optional held-out promotion gating. | [multi-shot-optimization.md](./docs/multi-shot-optimization.md) |
 | `runPromptEvolution`, `createCompositeMutator`, `createSandboxPool`, `createSandboxCodeMutator`, `MutationTelemetry`, `LineageRecorder`, `CostLedger`, `JsonlTrialCache` | Prompt + code evolution loops with bounded sandbox pools, durable JSONL telemetry, plateau-detecting composite mutators, crash-resumable trial cache. | §Evolution loop |
 | `reflective-mutation` (`buildReflectionPrompt`, `parseReflectionResponse`, `DEFAULT_MUTATION_PRIMITIVES`) | Trace-conditioned LLM mutator that reasons over top/bottom trials instead of blind rewrites. | inline JSDoc |

clients/python/README.md

@@ -140,7 +140,7 @@ All errors carry `.code` and `.details` (the structured payload from the server)
 
 ## Versioning
 
-This package is **version-locked** to the npm package. `tangle-agent-eval==0.18.0` ↔ `@tangle-network/agent-eval@0.18.0`. The two ship from the same git tag in the same CI workflow; if either fails to publish, neither does. Mismatched versions are a build-time error.
+This package is **version-locked** to the npm package. `tangle-agent-eval==0.19.0` ↔ `@tangle-network/agent-eval@0.19.0`. The two ship from the same git tag in the same CI workflow; if either fails to publish, neither does. Mismatched versions are a build-time error.
 
 `wire_version` is separate. It bumps only on breaking schema changes. Package versions can differ across releases as long as `wire_version` is the same.
 

clients/python/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "tangle-agent-eval"
-version = "0.18.0"
+version = "0.19.0"
 description = "Python client for @tangle-network/agent-eval — judge content against rubrics over HTTP or stdio RPC."
 readme = "README.md"
 requires-python = ">=3.10"

clients/python/src/tangle_agent_eval/__init__.py

@@ -39,7 +39,7 @@
     VersionResponse,
 )
 
-__version__ = "0.18.0"
+__version__ = "0.19.0"
 
 __all__ = [
     "Client",

docs/feature-guide.md

@@ -33,7 +33,7 @@ trying, and whether a change made them better or worse.
 | “Human feedback should become reusable eval data.” | `FeedbackTrajectory` | Captures approvals, rejections, edits, choices, metrics, and policy blocks. |
 | “Can this action run, or does it need approval?” | `evaluateActionPolicy` | Generic preflight for side effects, budgets, and required evidence. |
 | “I need train/dev/test/holdout examples.” | `Dataset` plus feedback trajectory conversion | Stable splits and contamination control. |
-| “Which prompt or signature wins?” | `PromptOptimizer`, `OptimizationLoop`, steering optimizers | Runs variants on scenarios and compares scores. |
+| “Which prompt or signature wins?” | `runMultiShotOptimization`, steering optimizers | Runs variants on scenarios and compares scores. |
 | “Improve a multi-turn agent over real task traces.” | `runMultiShotOptimization` | GEPA-style trajectory optimization with ASI and held-out promotion. |
 | “Improve prompts, then code if prompts plateau.” | `runPromptEvolution`, composite mutator, code mutator | Bounded evolution with telemetry and lineage. |
 | “Find why a regression happened.” | bisector, traces, run records | Narrows changes and preserves evidence. |
@@ -156,7 +156,7 @@ Store as `FeedbackTrajectory`, then derive:
 | Feedback data | `FeedbackTrajectory`, stores, converters | Human/environment labels | Domain adapters live in downstream repos. |
 | Action policy | `evaluateActionPolicy` | Approval/budget preflight | Blocks or labels actions before `act()`. |
 | Datasets | `Dataset`, holdout tools, canaries | Train/dev/test/holdout corpora | Keeps optimization honest. |
-| Optimization | `PromptOptimizer`, `OptimizationLoop`, steering optimizers | Prompt/signature comparison | Use held-out gates before promotion. |
+| Optimization | `runMultiShotOptimization`, steering optimizers | Prompt/signature comparison | Use held-out gates before promotion. |
 | Evolution | prompt/code mutators, sandbox pool, telemetry | Autoresearch and mutation loops | Use budgets and lineage; do not run unbounded. |
 | Telemetry | `TraceStore`, OTLP, file sinks | Audit and replay | Treat traces as evidence, not just logs. |
 | Reporting | summaries, pareto, cost tracker | Decision support | Useful for PRs, launch gates, research notes. |

docs/wire-protocol.md

@@ -96,7 +96,7 @@ GET /v1/version
 ```json
 {
   "package": "@tangle-network/agent-eval",
-  "version": "0.18.0",
+  "version": "0.19.0",
   "wireVersion": "1.0.0",
   "apiSurface": ["judge", "listRubrics", "version"]
 }

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tangle-network/agent-eval",
-  "version": "0.18.0",
+  "version": "0.19.0",
   "description": "Trace-first evaluation framework for Tangle agents. Core (spans, pipelines, sandbox harness, OTLP export), trust (dataset, red-team, calibration, behavior DSL), builder-of-builders (three-layer eval, resumable sessions, meta-runtime correlation), and frontier (meta-eval correlation study, Process Reward Modeling, bisector).",
   "homepage": "https://github.com/tangle-network/agent-eval#readme",
   "repository": {

src/code-mutator.ts

@@ -26,7 +26,7 @@
 
 import type {
   MutateAdapter,
-  PromptVariant,
+  EvolvableVariant,
   TrialResult,
   VariantAggregate,
 } from './prompt-evolution'
@@ -49,7 +49,7 @@ export interface CodeMutationOutcome {
   childId?: string
   /** Free-form one-liner: "tightened tool descriptions in forge-tools.ts". */
   description?: string
-  /** What the runner was trying to fix (carried into PromptVariant.rationale). */
+  /** What the runner was trying to fix (carried into EvolvableVariant.rationale). */
   rationale?: string
   /** Caller-defined diff payload. Mapped into the variant's payload by
    *  `toVariantPayload`; agent-eval treats it as opaque. */
@@ -67,7 +67,7 @@ export interface CodeMutationOutcome {
 
 export type CodeMutationRunner<T, P> = (args: {
   slot: PoolSlot<T>
-  parent: PromptVariant<P>
+  parent: EvolvableVariant<P>
   parentAggregate: VariantAggregate
   topTrials: TrialResult[]
   bottomTrials: TrialResult[]
@@ -83,25 +83,25 @@ export interface CreateSandboxCodeMutatorOpts<T, P> {
    * encode the diff however they want (file map, patch string, branch
    * ref, snapshot id) without agent-eval taking a stance.
    */
-  toVariantPayload(outcome: CodeMutationOutcome, parent: PromptVariant<P>): P
+  toVariantPayload(outcome: CodeMutationOutcome, parent: EvolvableVariant<P>): P
   /** Optional telemetry sinks. */
   mutationTelemetry?: MutationTelemetry
   costLedger?: CostLedger
   lineage?: LineageRecorder<P>
   /** Override id generation. Default: `${parent.id}.g${generation}.code.${i}`. */
-  childIdFor?(parent: PromptVariant<P>, generation: number, index: number): string
+  childIdFor?(parent: EvolvableVariant<P>, generation: number, index: number): string
   /** Default label for the variant (visible in reports). */
-  labelFor?(outcome: CodeMutationOutcome, parent: PromptVariant<P>, generation: number, index: number): string
+  labelFor?(outcome: CodeMutationOutcome, parent: EvolvableVariant<P>, generation: number, index: number): string
 }
 
 export function createSandboxCodeMutator<T, P>(
   opts: CreateSandboxCodeMutatorOpts<T, P>,
 ): MutateAdapter<P> {
   const childIdFor = opts.childIdFor
-    ?? ((parent: PromptVariant<P>, generation: number, index: number) =>
+    ?? ((parent: EvolvableVariant<P>, generation: number, index: number) =>
         `${parent.id}.g${generation}.code.${index}`)
   const labelFor = opts.labelFor
-    ?? ((outcome: CodeMutationOutcome, parent: PromptVariant<P>, _generation: number, index: number) =>
+    ?? ((outcome: CodeMutationOutcome, parent: EvolvableVariant<P>, _generation: number, index: number) =>
         outcome.description?.slice(0, 80) ?? `${parent.label} → code.${index}`)
 
   return {
@@ -136,7 +136,7 @@ export function createSandboxCodeMutator<T, P>(
         }
       })
 
-      const variants: PromptVariant<P>[] = []
+      const variants: EvolvableVariant<P>[] = []
       let index = 0
       for (const outcome of outcomes) {
         const childId = outcome.childId ?? childIdFor(parent, generation, index)
@@ -164,7 +164,7 @@ export function createSandboxCodeMutator<T, P>(
         }
 
         if (outcome.ok) {
-          const variant: PromptVariant<P> = {
+          const variant: EvolvableVariant<P> = {
             id: childId,
             payload: opts.toVariantPayload(outcome, parent),
             generation,

src/composite-mutator.ts

@@ -18,7 +18,7 @@
 
 import type {
   MutateAdapter,
-  PromptVariant,
+  EvolvableVariant,
   TrialResult,
   VariantAggregate,
 } from './prompt-evolution'
@@ -42,7 +42,7 @@ export interface CreateCompositeMutatorOpts<P> {
 }
 
 interface MutateArgs<P> {
-  parent: PromptVariant<P>
+  parent: EvolvableVariant<P>
   parentAggregate: VariantAggregate
   topTrials: TrialResult[]
   bottomTrials: TrialResult[]
@@ -91,7 +91,7 @@ export function createCompositeMutator<P>(opts: CreateCompositeMutatorOpts<P>):
   }
 
   return {
-    async mutate(args: MutateArgs<P>): Promise<PromptVariant<P>[]> {
+    async mutate(args: MutateArgs<P>): Promise<EvolvableVariant<P>[]> {
       const { mode, reason } = pickMode(args)
       opts.onPolicyDecision?.({ generation: args.generation, chose: mode, reason })