feat(review): support additional reviewer harnesses#2306
Open
anirudh5harma wants to merge 2 commits into
Open
Conversation
ac03b8c to
c0049ae
Compare
anirudh5harma
commented
Jun 30, 2026
Author
There was a problem hiding this comment.
The live run exposed daemon-environment inheritance and interactive-PTY input issues, which were corrected in df4e3df. After the fix, local lint, build, vet, race tests, frontend typecheck, 40 test files / 371 tests, and the production renderer build passed.
Collaborator
|
Can you please add testing details of running codex and opencode agent harness for reviewer? |
Author
|
@neversettle17-101 added a full end-to-end testing recipe to the PR description under "Testing Codex and OpenCode reviewer harnesses end-to-end". Covers:
Let me know if you'd like a screen recording of the live runs as well. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reviewer sessions can now use the two additional harnesses requested in #2298, including automatic reuse of a worker's harness when no reviewer override is configured. Both adapters reuse the existing worker launchers while applying reviewer-specific read-only controls, and Project Settings exposes the expanded reviewer choices.
Validation
npm run lintcd backend && go build ./... && go vet ./...cd backend && go test -race ./...cd frontend && npm run typecheckcd frontend && npm test(40 files, 371 tests)cd frontend && npx vite build --config vite.renderer.config.tsPost-Deploy Monitoring & Validation
no reviewer adapter,reviewer command, andreviewer runtimeafter review launches.running, open a reviewer pane, and submit a verdict without modifying the worker checkout.Closes #2298.
Testing Codex and OpenCode reviewer harnesses end-to-end
Concrete recipe for running a review with each new harness against a live daemon, requested in this comment.
Prerequisites
codexand/oropencodeCLIs installed and on$PATH(verify withcodex --version/opencode --version)ao startor the desktop app)1. Set the reviewer harness for a project
Two ways:
Settings UI —
Project → Settings → Reviewers → Default reviewer agent. The Select now listscodexandopencodealongsideclaude-code.CLI — write the config directly:
Verify it persisted:
2. Trigger a review on the worker's open PR
The daemon resolves the reviewer harness via
ProjectConfig.ResolveReviewerHarness, picks the matching adapter (reviewer/codexorreviewer/opencode), and spawns a reviewer pane in the same workspace.3. Verify the reviewer pane launched with the right sandbox
Codex — the adapter inserts
--sandbox read-onlyand forwards AO location env vars via Codex's-c shell_environment_policy.set.*. Inspect the live process:Attempt an edit inside the codex pane (e.g.
:!echo x >> README.md) — Codex refuses because the sandbox is read-only.git diff,gh api, andao review submitremain available.OpenCode — the adapter injects the permission policy via the
OPENCODE_CONFIG_CONTENTenv var. Inspect:Inside the OpenCode pane: read/grep/glob and the listed bash commands work; every other tool and bash command is denied.
4. Verify the verdict round-trip
The reviewer pane runs
ao review submit --review-id <id> --reviews <json>. Daemon writes the verdict and emits a notification:Unit + build coverage already in CI
backend/internal/adapters/reviewer/codex/codex_test.go:TestReviewCommandUsesReadOnlySandbox— asserts--sandbox read-onlyis inserted before the prompt argument and each AO_* env var becomes a-c shell_environment_policy.set.*entryTestReviewMessageReturnsTaskPrompt— confirms a follow-up message reuses the same task promptbackend/internal/adapters/reviewer/opencode/opencode_test.go:TestReviewCommandUsesReadOnlyPermissionPolicy— asserts the exact policy JSON travels via theOPENCODE_CONFIG_CONTENTenv var and the system prompt is folded into the promptTestReviewMessageReturnsTaskPromptbackend/internal/domain/projectconfig_test.go—Validatetable rows reject configs that name an unsupported reviewer harness and accept the two new onesfrontend/src/renderer/components/ProjectSettingsForm.test.tsx— Reviewer<Select>round-tripscodexandopencodethrough saveRun locally: