Commit 819386a
committed
Optimize lexer hot paths with frozenset lookups
Replace string method calls (isascii, isalnum, isdigit) with
frozenset membership checks for character classification. This
reduces function call overhead in the lexer's hot paths.
Key changes:
- character_classes.py: Use frozenset lookups instead of str methods
- lexer.py: Inline frozenset checks in read_name, read_next_token
- Export NAME_CONTINUE, NAME_START, DIGITS, WHITESPACE constants
Performance improvement: ~25% faster parsing on large queries.
Profiled read_name drops from 30% to not-in-top-15 of parse time.1 parent fef26da commit 819386a
2 files changed
Lines changed: 41 additions & 19 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
2 | | - | |
3 | | - | |
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
4 | 27 | | |
5 | 28 | | |
6 | 29 | | |
7 | 30 | | |
8 | 31 | | |
9 | 32 | | |
10 | 33 | | |
11 | | - | |
| 34 | + | |
12 | 35 | | |
13 | 36 | | |
14 | 37 | | |
15 | 38 | | |
16 | 39 | | |
17 | 40 | | |
18 | 41 | | |
19 | | - | |
| 42 | + | |
20 | 43 | | |
21 | 44 | | |
22 | 45 | | |
23 | 46 | | |
24 | 47 | | |
25 | 48 | | |
26 | 49 | | |
27 | | - | |
| 50 | + | |
28 | 51 | | |
29 | 52 | | |
30 | 53 | | |
31 | 54 | | |
32 | 55 | | |
33 | 56 | | |
34 | 57 | | |
35 | | - | |
| 58 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | | - | |
| 10 | + | |
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| |||
109 | 109 | | |
110 | 110 | | |
111 | 111 | | |
112 | | - | |
| 112 | + | |
113 | 113 | | |
114 | 114 | | |
115 | 115 | | |
| |||
138 | 138 | | |
139 | 139 | | |
140 | 140 | | |
141 | | - | |
| 141 | + | |
142 | 142 | | |
143 | 143 | | |
144 | | - | |
| 144 | + | |
| 145 | + | |
145 | 146 | | |
146 | 147 | | |
147 | 148 | | |
| |||
204 | 205 | | |
205 | 206 | | |
206 | 207 | | |
207 | | - | |
| 208 | + | |
208 | 209 | | |
209 | 210 | | |
210 | 211 | | |
| |||
231 | 232 | | |
232 | 233 | | |
233 | 234 | | |
234 | | - | |
| 235 | + | |
235 | 236 | | |
236 | 237 | | |
237 | 238 | | |
| |||
248 | 249 | | |
249 | 250 | | |
250 | 251 | | |
251 | | - | |
| 252 | + | |
252 | 253 | | |
253 | 254 | | |
254 | 255 | | |
| |||
259 | 260 | | |
260 | 261 | | |
261 | 262 | | |
262 | | - | |
| 263 | + | |
263 | 264 | | |
264 | 265 | | |
265 | 266 | | |
| |||
452 | 453 | | |
453 | 454 | | |
454 | 455 | | |
455 | | - | |
456 | | - | |
457 | | - | |
458 | | - | |
| 456 | + | |
| 457 | + | |
459 | 458 | | |
460 | 459 | | |
461 | 460 | | |
| |||
0 commit comments