feat(frontend): Run-on modes in the evaluator creation drawer (shared controls)#4557
Merged
Merged
Conversation
The Run-on selector (test case / app output / trace) was only wired into the
full-page evaluator playground. The evaluator-creation drawer still hardcoded
`runDisabled={!hasAppConnected}` and only showed the test-set dropdown after an
app was connected, so it forced the user to pick an app even when they wanted to
run the evaluator directly on a test case.
Rather than copy the run-on wiring into the drawer (a fourth duplicate), extract
the shared logic the page and drawer were already duplicating:
- useEvaluatorRunControls(): app adapter, app-select handler, run-on mode +
handlePickRunOn, and the runDisabled gate (runOnMode === 'app' && !appConnected).
- EvaluatorRunControls: the run-on selector + app picker + disconnect + test-set
cluster, shared by the page header and the drawer header so they can't drift.
The page is behavior-preserving; the drawer gains all three modes, the run-on
selector, a disconnect affordance, and an always-available test-set dropdown.
This also removes the adapter/handleAppSelect/evaluator-node triplication across
the page body, drawer header, and drawer body.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
dosubot
Bot
added
the
size:L
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
Railway Preview Environment
|
The creation drawer renders inside EvaluationRunsTableStoreProvider, a scoped jotai store that mirrors only a handful of global atoms. The playground state, however, runs on the default store (the playground package uses getDefaultStore() throughout). So in the drawer the run-on mode was read/written in the scoped store while the playground lived in the default store — the two split, and switching to test-case mode never reached the run panel: it stayed stuck on the 'Select an app' empty state. Read and write all run-on / playground atoms through getDefaultStore() in useEvaluatorRunControls, mirroring the existing workaround in usePreviewVariantConfig and TestsetCells. On the full page (no scoped store) this is a no-op; in the drawer it aligns run-on state with the playground so test-case mode shows the inputs/outputs as it does on the page.
dosubot
Bot
added
size:XL
ardaerzin
approved these changes
Jun 8, 2026
mmabrouk
changed the base branch from
fe-fix/app-workflow-router-unification-regression-fix
to
release/v0.102.0
The evaluator drawer rendered by WorkflowRevisionDrawerWrapper reimplemented
the run panel gate as `runDisabled={!hasAppConnected}`, ignoring the run-on
mode. So switching its Run-on selector to 'test case' updated the header while
the panel kept showing the 'Select an app' empty state and demanding an app —
the page and creation drawer respected the mode, only this third surface didn't.
Route it through the shared useEvaluatorRunControls hook (+ SelectAppEmptyState
and the prop-less EvaluatorPlaygroundHeader), the same wiring the page and the
creation drawer use, so the gate is `runOnMode === 'app' && !hasAppConnected`
everywhere and the three surfaces can't drift again. Removes this drawer's
duplicated app adapter / app-select / run-gate logic.
Also drop the getDefaultStore() patch from useEvaluatorRunControls: runtime
debugging proved these surfaces are not in a scoped store (the drawer that was
broken is WorkflowRevisionDrawerWrapper, not the scoped-store CreateEvaluator
drawer), so the override was a no-op based on a wrong hypothesis.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The Run on selector (test case / app output / trace) was only wired into the full-page evaluator playground. The evaluator-creation drawer still hardcoded
runDisabled={!hasAppConnected}and only showed the test-set dropdown after an app was connected — so in the drawer you were forced to pick an app even when you wanted to run the evaluator directly on a test case. The drawer had silently drifted out of sync with the page.What
Rather than paste the run-on wiring into the drawer (a fourth copy), this extracts the logic the page and drawer were already duplicating and shares it:
useEvaluatorRunControls()— one hook for the app adapter, app-select handler, run-on mode +handlePickRunOn, and the run gate (runDisabled = runOnMode === "app" && !hasAppConnected).EvaluatorRunControls— the run-on selector + app picker + disconnect affordance + test-set dropdown, as one cluster used by both the page header and the drawer header, so they can't diverge again.Result:
appWorkflowAdapter/handleAppSelect/ evaluator-node-lookup triplication across the page body, drawer header, and drawer body.Net: 218 insertions / 274 deletions across 5 files (2 new, 3 slimmed).
Notes
runOnModestays persisted per project (shared by page and drawer); the per-evaluator question is tracked separately for a later PR, as discussed.runDisabledonly manifests where the run panel renders (the page and the expanded drawer); the collapsed/config-only drawer ignores it, unchanged.Stacked on
Based on
fe-fix/app-workflow-router-unification-regression-fix(the merged evaluator-playground branch, which already contains the page-side run-on feature from #4553).Test plan