perf: use lookup table in splitIntoPotentialTokens

[xiaoxiaojx]

Summary

• Replace multi-comparison chains in splitIntoPotentialTokens with a 128-byte Uint8Array bitmask lookup table
• Phase 1 previously checked 4 conditions (cc !== 10/59/123/125) and phase 2 checked 6 conditions (cc === 59/32/123/125/13/9) per character — now both are a single indexed read + bitwise AND

Benchmark results (ops/s, higher is better)

Benchmark	Before	After	Change
splitIntoPotentialTokens: fixture	533	574	+7.7%
splitIntoPotentialTokens: big source	392	474	+20.9%
original-source: streamChunks()	904	983	+8.7%
original-source: streamChunks() (big)	334	419	+25.4%
original-source: map()	628	669	+6.5%
original-source: sourceAndMap()	626	667	+6.5%
concat-source: streamChunks() (mixed)	812	882	+8.6%
concat-source: sourceAndMap()	586	623	+6.3%
realistic cold sourceAndMap()	864	901	+4.3%
realistic cold map()	863	899	+4.2%
realistic streamChunks(columns on)	1,154	1,208	+4.7%

No regressions observed. All 89,869 tests pass.

Test plan

• yarn lint passes
• npx jest --no-coverage — 89,869 tests pass
• Benchmark verified with npm run benchmark

🤖 Generated with Claude Code

[changeset-bot]

🦋 Changeset detected

Latest commit: 04b8a57

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
webpack-sources	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[codspeed-hq]

Merging this PR will improve performance by 23.21%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 13 improved benchmarks
❌ 21 regressed benchmarks
✅ 177 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Memory	prefix-source memory: source() allocates rewritten string	2 KB	1,043.8 KB	-99.81%
❌	Memory	source-map-source memory: new SourceMapSource(with inner map)	1.5 KB	32.9 KB	-95.3%
❌	Memory	source-map-source memory: map({ columns: true })	880 B	9,544 B	-90.78%
❌	Memory	concat-source memory: new ConcatSource(...children)	2.5 KB	8.4 KB	-69.7%
❌	Memory	compat-source memory: new CompatSource(sourceLike)	96.8 KB	242.4 KB	-60.08%
❌	Memory	compat-source memory: delegated source() + map() through wrapper	11.9 KB	22.3 KB	-46.82%
❌	Memory	original-source memory: map({ columns: false }) line-only mappings	772.3 KB	1,378 KB	-43.96%
❌	Memory	original-source memory: sourceAndMap({ columns: true })	1.3 MB	2 MB	-34.88%
❌	Memory	concat-source memory: map({ columns: true }) composes child maps	3.3 MB	4.4 MB	-24.06%
❌	Memory	clear-cache memory: unique tasks (clearCache default)	2.8 MB	3.6 MB	-22.83%
❌	Memory	concat-source memory: source() concatenates children	276.9 KB	356.6 KB	-22.35%
❌	Simulation	helpers/splitIntoPotentialTokens: fixture	10.9 ms	13.9 ms	-21.59%
❌	Simulation	helpers/splitIntoPotentialTokens: big source	16.7 ms	21.1 ms	-21.11%
❌	Memory	raw-source memory: new RawSource(buffer)	784 B	912 B	-14.04%
❌	Simulation	original-source: streamChunks({finalSource:true})	10.1 ms	11.6 ms	-12.87%
❌	Simulation	original-source: streamChunks()	10.2 ms	11.7 ms	-12.81%
❌	Simulation	original-source: streamChunks() (big source)	30.6 ms	35.1 ms	-12.71%
❌	Memory	clear-cache memory: unique tasks (clearCache maps + parsedMap, keep source)	3.6 MB	4.1 MB	-12.23%
❌	Simulation	prefix-source: streamChunks()	6.1 ms	6.9 ms	-10.91%
❌	Simulation	concat-source: streamChunks({finalSource:true})	12.9 ms	14.4 ms	-10.41%
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing perf/split-into-potential-tokens-lookup-table (04b8a57) with main (e02802e)}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.59%. Comparing base (e02802e) to head (04b8a57).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #240   +/-   ##
=======================================
  Coverage   97.58%   97.59%           
=======================================
  Files          25       25           
  Lines        2069     2076    +7     
  Branches      668      664    -4     
=======================================
+ Hits         2019     2026    +7     
  Misses         47       47           
  Partials        3        3           

Flag	Coverage Δ
integration	97.59% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.