Skip to content

perf(visualizer): optimize actionSpace lookups in prompt-input#2339

Closed
vincerevu wants to merge 4 commits intoweb-infra-dev:mainfrom
vincerevu:perf/optimize-actionspace-lookups-15008476720627830974
Closed

perf(visualizer): optimize actionSpace lookups in prompt-input#2339
vincerevu wants to merge 4 commits intoweb-infra-dev:mainfrom
vincerevu:perf/optimize-actionspace-lookups-15008476720627830974

Conversation

@vincerevu
Copy link
Copy Markdown
Contributor

What:
The optimization implemented converts the actionSpace array into a pre-computed Map keyed by action.name and action.interfaceAlias using a single useMemo hook in the PromptInput component. It replaces all occurrences of actionSpace.find(...) with actionMap.get(selectedType). It also updates the exported isRunButtonEnabled utility function signature to accept the map directly, eliminating its internal duplicate lookups.

Why:
Previously, the code performed O(n) array .find() lookups across over a dozen different useMemo, useCallback, and render paths. For large action spaces or high-frequency render passes (like user typing in the prompt input), this caused redundant iterations that were computationally expensive. Converting this to an O(1) Map lookup eliminates these repeated loop iterations entirely.

Measured Improvement:
In a focused benchmark with a 1,000-item action space simulating 100,000 evaluations, the baseline array-based approach took ~1430ms. Using the optimized actionMap approach, the time dropped to ~65ms, representing a massive ~22x speedup over the baseline.

Convert `actionSpace` array to an `actionMap` using a `useMemo` hook to turn 10+ repeated O(n) `.find()` lookups across various hooks into O(1) `.get()` lookups.
Update `isRunButtonEnabled` utility signature to accept the `actionMap` instead of `actionSpace` array, removing redundant internal iterations. This results in a ~22x execution speed improvement in benchmark tests for the button evaluation logic.
Convert `actionSpace` array to an `actionMap` using a `useMemo` hook to turn 10+ repeated O(n) `.find()` lookups across various hooks into O(1) `.get()` lookups.
Update `isRunButtonEnabled` utility signature to accept the `actionMap` instead of `actionSpace` array, removing redundant internal iterations. This results in a ~22x execution speed improvement in benchmark tests for the button evaluation logic.
Convert `actionSpace` array to an `actionMap` using a `useMemo` hook to turn 10+ repeated O(n) `.find()` lookups across various hooks into O(1) `.get()` lookups.
Update `isRunButtonEnabled` utility signature to accept the `actionMap` instead of `actionSpace` array, removing redundant internal iterations. This results in a ~22x execution speed improvement in benchmark tests for the button evaluation logic.
Convert `actionSpace` array to an `actionMap` using a `useMemo` hook to turn 10+ repeated O(n) `.find()` lookups across various hooks into O(1) `.get()` lookups.
Update `isRunButtonEnabled` utility signature to accept the `actionMap` instead of `actionSpace` array, removing redundant internal iterations. This results in a ~22x execution speed improvement in benchmark tests for the button evaluation logic.
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5a483f9b6e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +104 to +105
if (action.name) map.set(action.name, action);
if (action.interfaceAlias) map.set(action.interfaceAlias, action);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve first-match precedence when indexing actionSpace

Building actionMap with unconditional map.set(...) makes duplicate name/interfaceAlias keys resolve to the last action, while the rest of the execution path still resolves by first match (executeAction in packages/playground/src/common.ts uses actionSpace.find(...)). This introduces a behavioral mismatch: for action spaces that include duplicates (e.g., default actions plus same-alias custom actions), the prompt UI/validation now reads one action schema but runtime dispatch executes a different action, which can reject parameters or run the wrong behavior.

Useful? React with 👍 / 👎.

@quanru
Copy link
Copy Markdown
Collaborator

quanru commented Apr 21, 2026

Thanks for the contribution, @vincerevu — really appreciate the time you put into measuring this.

After reviewing, we've decided not to move forward with this change. A few reasons:

  1. Real-world workload doesn't match the benchmark. In practice actionSpace holds on the order of a few dozen entries, and all .find(...) calls here live inside useMemos that only re-evaluate when selectedType / actionSpace changes — not on every keystroke. At n ≈ few dozen, .find vs Map.get is not user-perceptible. The 22× figure comes from n=1000 × 100k iterations, which isn't representative.
  2. Subtle semantics change without test coverage. Array.prototype.find returns the first match; Map.set keeps the last write. Building the map with both name and interfaceAlias as keys means that if two actions ever collide across those fields (A.name === B.interfaceAlias), the lookup result changes. Today that collision is unlikely, but there's no invariant check or test locking it in.
  3. Can be a net-negative at small n. Every time actionSpace's reference changes, actionMap rebuilds and every downstream useMemo / useCallback that depends on it re-runs. For a small array, that's more work than the original .find calls would have been.
  4. Partial coverage. There are still several actionSpace.find(...) call sites in prompt-input/index.tsx on main that this PR doesn't touch, and the branch is currently conflicting.

If you'd like to contribute a perf improvement here, a more impactful direction would be profiling actual render hot paths in PromptInput (form state, re-render cascades on typing) rather than optimizing these cold lookups. Happy to help point at those if you're interested.

Closing for now — thanks again!

@quanru quanru closed this Apr 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants