[maestro] Add painter tool for image generation and editing#823
Closed
haasonsaas wants to merge 1 commit into
Closed
[maestro] Add painter tool for image generation and editing#823haasonsaas wants to merge 1 commit into
haasonsaas wants to merge 1 commit into
Conversation
Painter is the visual counterpart to the Oracle: a tool the main agent invokes to generate or edit images via an image model (default gpt-image-2), mirroring the Oracle's createTool shape and registration footprint. Why: Maestro had no image surface, so UI mockups, app icons, and screenshot redaction all required leaving the agent thread. Amp ships this as a closed feature; this gives Maestro an open equivalent that composes with the existing Conductor browser tooling. Design: - Outputs are persisted to disk and returned as absolute paths, never inlined as base64, so transcripts stay small and images are @-mentionable. - Thin ImageProvider abstraction with an honest supports map; only the OpenAI provider is shipped (generate + masked edit). The interface lets FLUX/Imagen land later without touching the tool. - Combines the tool abort signal with a hard timeout without relying on AbortSignal.any, for lib-target portability. - Retries only on transient network/rate-limit faults (errno + message), never on validation or content-policy rejections. Wiring: lazy-registry, tools/index (codingTools + toolRegistry + category doc), subagent-specs (advanced category; denied from explorer like oracle), env-vars (MAESTRO_PAINTER_MODEL/BASE_URL/OUTPUT_DIR/TIMEOUT_MS), and the tool-surface smoke contract (codingCount 27 -> 28). Tests cover env resolution, timeout parsing, the transient classifier, the persist round-trip, schema/guard rejections, and the OpenAI provider's decode/write/mask paths via an injected stub client. Writing the classifier test surfaced a real gap: errno codes like ETIMEDOUT were not matched, now fixed via the err.code property.
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
Bugbot needs on-demand usage enabledBugbot uses usage-based billing for this team and requires on-demand usage to be enabled. A team admin can enable on-demand usage in the Cursor dashboard. |
Contributor
|
This PR changes mirrored Maestro source files in the public repo, but it does not link the matching private source-of-truth PR. Add one of these to the PR body, then re-run the check:
Mirrored files touched:
|
Contributor
Author
|
Closing — this was opened against the public mirror, which is one-way synced from the internal repo. Retargeted to the real source of truth: https://github.com/evalops/maestro-internal/pull/2864 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a Painter tool — the visual counterpart to the Oracle — that generates or edits images via an image model (default
gpt-image-2). It mirrors the Oracle'screateToolshape and registration footprint.Why
Maestro had no image surface, so UI mockups, app icons, and screenshot redaction all required leaving the agent thread. Amp ships this as a closed feature; this gives Maestro an open equivalent that also composes with the existing Conductor browser tooling (e.g. screenshot a bug → redact → attach to a PR).
Surface
src/services/image-providers/{types,openai,index}.ts— provider abstraction with an honestsupportsmap; OpenAI impl (generate + masked edit) using the already-presentopenaiSDK.src/tools/painter.ts— the tool. Outputs persisted to disk and returned as absolute paths, never inlined as base64 (keeps transcripts small, images stay@-mentionable).lazy-registry,tools/index(codingTools+toolRegistry),subagent-specs(advancedcategory; denied fromexplorerlike oracle),env-vars(MAESTRO_PAINTER_MODEL/BASE_URL/OUTPUT_DIR/TIMEOUT_MS), and the tool-surface smoke contract (codingCount27 → 28).docs/PAINTER.md— setup, env table, examples, gpt-image-2 notes (no transparency), and explicit v1 limits.Design decisions
supportsmap keeps generate-vs-edit asymmetry honest; FLUX/Imagen can land later without touching the tool.profiles.tsentry. Image gen doesn't vary by capability mode the way reasoning does; the model resolves viaMAESTRO_PAINTER_MODEL→gpt-image-2.AbortSignal.any) for lib-target portability.Tests
Unit tests (
test/tools/painter.test.ts,test/services/image-providers/openai.test.ts— 24 cases) cover env resolution, timeout parsing, the transient-error classifier, the persist round-trip, schema/guard rejections, and the OpenAI provider's decode/write/mask paths via an injected stub client. Writing the classifier test surfaced a real gap — errno codes likeETIMEDOUTweren't matched — now fixed via theerr.codeproperty.Verification
tsc -b tsconfig.build.json— cleanbiome check— clean (no fixes pending)viteston the two new test files — 24/24 passingtool-surface-smoke-evals— passing (contract updated for the new tool)Not run this session: the full
npx nx run maestro:testsuite; relying on CI for the broader sweep.Out of scope (follow-ups)
MAESTRO_PAINTER_MAX_COST_CENTS)