Skip to content

perf: use a dict for linked package store directory membership checks#2908

Merged
jbedard merged 1 commit into
aspect-build:mainfrom
jbedard:perf-linked-store-dirs-dict
Jul 9, 2026
Merged

perf: use a dict for linked package store directory membership checks#2908
jbedard merged 1 commit into
aspect-build:mainfrom
jbedard:perf-linked-store-dirs-dict

Conversation

@jbedard

@jbedard jbedard commented Jul 9, 2026

Copy link
Copy Markdown
Member

The 'not in' check against a list is O(direct deps) for each entry in the transitive npm_package_store_infos of the package, making the linking loop quadratic for packages with many deps.

Changes are visible to end-users:no

Test plan

  • Covered by existing test cases

@jbedard jbedard requested review from acozzette and dzbarsky July 9, 2026 04:05
@jbedard jbedard enabled auto-merge (squash) July 9, 2026 04:08

@dzbarsky dzbarsky left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you actually measure a win? I feel like linear scan is usually faster than dict for up to 10ish elements so this might be a loss in aggregate

Comment thread npm/private/npm_package_store.bzl Outdated
@jbedard

jbedard commented Jul 9, 2026

Copy link
Copy Markdown
Member Author

Did you actually measure a win? I feel like linear scan is usually faster than dict for up to 10ish elements so this might be a loss in aggregate

I can investigate this specific one more and try both.

@jbedard jbedard disabled auto-merge July 9, 2026 17:29
@jbedard jbedard force-pushed the perf-linked-store-dirs-dict branch from 2770b7a to eb79a6e Compare July 9, 2026 17:29
@jbedard

jbedard commented Jul 9, 2026

Copy link
Copy Markdown
Member Author

I feel like linear scan is usually faster than dict for up to 10ish elements so this might be a loss in aggregate

A robot says:

the dict is win-or-tie everywhere — membership at D=3 (the linear scan's best case) is a tie, and inserts favor the dict ~2x because list.append() is a method call while d[k] = None isn't — so there's no size at which the list wins in aggregate.

@jbedard jbedard force-pushed the perf-linked-store-dirs-dict branch from eb79a6e to 292a834 Compare July 9, 2026 20:48
The 'not in' check against a list is O(direct deps) for each entry in
the transitive npm_package_store_infos of the package, making the
linking loop quadratic for packages with many deps.
@jbedard jbedard force-pushed the perf-linked-store-dirs-dict branch from 292a834 to 452dfa0 Compare July 9, 2026 20:54
@jbedard jbedard merged commit d82417a into aspect-build:main Jul 9, 2026
153 checks passed
@jbedard jbedard deleted the perf-linked-store-dirs-dict branch July 9, 2026 22:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants