Skip to content

[maestro] Add painter tool for image generation and editing#823

Closed
haasonsaas wants to merge 1 commit into
evalops:mainfrom
haasonsaas:feat/painter
Closed

[maestro] Add painter tool for image generation and editing#823
haasonsaas wants to merge 1 commit into
evalops:mainfrom
haasonsaas:feat/painter

Conversation

@haasonsaas

Copy link
Copy Markdown
Contributor

What

Adds a Painter tool — the visual counterpart to the Oracle — that generates or edits images via an image model (default gpt-image-2). It mirrors the Oracle's createTool shape and registration footprint.

Why

Maestro had no image surface, so UI mockups, app icons, and screenshot redaction all required leaving the agent thread. Amp ships this as a closed feature; this gives Maestro an open equivalent that also composes with the existing Conductor browser tooling (e.g. screenshot a bug → redact → attach to a PR).

Surface

  • src/services/image-providers/{types,openai,index}.ts — provider abstraction with an honest supports map; OpenAI impl (generate + masked edit) using the already-present openai SDK.
  • src/tools/painter.ts — the tool. Outputs persisted to disk and returned as absolute paths, never inlined as base64 (keeps transcripts small, images stay @-mentionable).
  • Wired into lazy-registry, tools/index (codingTools + toolRegistry), subagent-specs (advanced category; denied from explorer like oracle), env-vars (MAESTRO_PAINTER_MODEL/BASE_URL/OUTPUT_DIR/TIMEOUT_MS), and the tool-surface smoke contract (codingCount 27 → 28).
  • docs/PAINTER.md — setup, env table, examples, gpt-image-2 notes (no transparency), and explicit v1 limits.

Design decisions

  • Paths, not base64. Transcript bloat is the failure mode to avoid.
  • Thin provider abstraction, OpenAI-only for v1. The supports map keeps generate-vs-edit asymmetry honest; FLUX/Imagen can land later without touching the tool.
  • No profiles.ts entry. Image gen doesn't vary by capability mode the way reasoning does; the model resolves via MAESTRO_PAINTER_MODELgpt-image-2.
  • Mask = file path only in v1 (bounding-box→PNG synthesis deferred).
  • Abort-signal combining done manually (no AbortSignal.any) for lib-target portability.

Tests

Unit tests (test/tools/painter.test.ts, test/services/image-providers/openai.test.ts — 24 cases) cover env resolution, timeout parsing, the transient-error classifier, the persist round-trip, schema/guard rejections, and the OpenAI provider's decode/write/mask paths via an injected stub client. Writing the classifier test surfaced a real gap — errno codes like ETIMEDOUT weren't matched — now fixed via the err.code property.

Verification

  • tsc -b tsconfig.build.json — clean
  • biome check — clean (no fixes pending)
  • vitest on the two new test files — 24/24 passing
  • tool-surface-smoke-evals — passing (contract updated for the new tool)

Not run this session: the full npx nx run maestro:test suite; relying on CI for the broader sweep.

Out of scope (follow-ups)

  • Bounding-box → mask synthesis
  • TUI inline image rendering (iTerm2/sixel/kitty)
  • Per-run cost ceiling (MAESTRO_PAINTER_MAX_COST_CENTS)
  • Additional providers (FLUX, Imagen)

Painter is the visual counterpart to the Oracle: a tool the main agent
invokes to generate or edit images via an image model (default gpt-image-2),
mirroring the Oracle's createTool shape and registration footprint.

Why: Maestro had no image surface, so UI mockups, app icons, and screenshot
redaction all required leaving the agent thread. Amp ships this as a closed
feature; this gives Maestro an open equivalent that composes with the
existing Conductor browser tooling.

Design:
- Outputs are persisted to disk and returned as absolute paths, never
  inlined as base64, so transcripts stay small and images are @-mentionable.
- Thin ImageProvider abstraction with an honest supports map; only the
  OpenAI provider is shipped (generate + masked edit). The interface lets
  FLUX/Imagen land later without touching the tool.
- Combines the tool abort signal with a hard timeout without relying on
  AbortSignal.any, for lib-target portability.
- Retries only on transient network/rate-limit faults (errno + message),
  never on validation or content-policy rejections.

Wiring: lazy-registry, tools/index (codingTools + toolRegistry + category
doc), subagent-specs (advanced category; denied from explorer like oracle),
env-vars (MAESTRO_PAINTER_MODEL/BASE_URL/OUTPUT_DIR/TIMEOUT_MS), and the
tool-surface smoke contract (codingCount 27 -> 28).

Tests cover env resolution, timeout parsing, the transient classifier, the
persist round-trip, schema/guard rejections, and the OpenAI provider's
decode/write/mask paths via an injected stub client. Writing the classifier
test surfaced a real gap: errno codes like ETIMEDOUT were not matched, now
fixed via the err.code property.
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.

@cursor

cursor Bot commented Jul 14, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@github-actions

Copy link
Copy Markdown
Contributor

This PR changes mirrored Maestro source files in the public repo, but it does not link the matching private source-of-truth PR.

Add one of these to the PR body, then re-run the check:

  • https://github.com/evalops/maestro-internal/pull/<number>
  • evalops/maestro-internal#<number>
  • maestro-internal#<number>

Mirrored files touched:

  • docs/PAINTER.md
  • evals/tools/surface-smoke-cases.json
  • src/agent/subagent-specs.ts
  • src/config/env-vars.ts
  • src/services/image-providers/index.ts
  • src/services/image-providers/openai.ts
  • src/services/image-providers/types.ts
  • src/tools/index.ts
  • src/tools/lazy-registry.ts
  • src/tools/painter.ts
  • test/services/image-providers/openai.test.ts
  • test/tools/painter.test.ts

@haasonsaas

Copy link
Copy Markdown
Contributor Author

Closing — this was opened against the public mirror, which is one-way synced from the internal repo. Retargeted to the real source of truth: https://github.com/evalops/maestro-internal/pull/2864

@haasonsaas haasonsaas closed this Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant