Commit 2089525
committed
[wasm-split] Do multi-split at once
This does multi-splitting of modules at once, rather than splitting them
one by one by doing 2-way split n times. Previously when we did
multi-splitting we split the 1st module as the "secondary" module
assuming all other functions belonging to 2nd~nth modules as "primary"
module. And then we repeat the same task for the 2nd module, assuming
3rd~nth module functions belong to the "primary" module. This
unnecessarily repeated some tasks that could have been done once.
This reduces the running time on a reproducer provided by @biggs0125
before (to fix #7725) from 236s to 88s, reducing it by around 63%.
Some side-products of this PR are:
- Now we only create a single table to host placeholders (or `ref.null`s
in case of `--no-placeholders`) even when reference-types is enabled.
Previously we created a table per secondary module, resulting in n
tables.
- The names of trampoline functions have been changed in the tests, but
semantically they are the same. (e.g. in
`test/lit/wasm-split/multi-split.wast`) The reason for the change is,
previously we split modules one by one, by the time we split the first
module, it assumed functions belonging to other secondary modules were
primary functions, but they later changed to trampolines as well. Now
they are all named as trampolines, arguably enhacing readability.
---
Some detailed analysis run using the reproducer of #7725, a case where
we split a module into 301 (1 primary + 300 secondary) modules:
- Before this PR:
Time: 236.8s
Task breakdown:
```
Task Total Time (ms) Percentage
---------------------------------------------------------------------------
shareImportableItems 62661.1860 28.24%
classifyFunctions 42366.7451 19.09%
removeUnusedSecondaryElements 33083.6602 14.91%
indirectReferencesToSecondaryFunctions 27852.3143 12.55%
indirectCallsToSecondaryFunctions 25091.4263 11.31%
moveSecondaryFunctions 14159.9166 6.38%
writeModule_secondary 9331.1667 4.20%
setupTablePatching 3099.9597 1.40%
initExportedPrimaryFuncs 1657.0465 0.75%
writeModule_primary 901.6800 0.41%
exportImportCalledPrimaryFunctions 892.0132 0.40%
thunkExportedSecondaryFunctions 826.8599 0.37%
initSecondary 0.2241 0.00%
---------------------------------------------------------------------------
Overall Total 221924.1985 100.00%
```
- After this PR:
Time : 88.40207334437098
Task breakdown:
```
Task Total Time (ms) Percentage
---------------------------------------------------------------------------
shareImportableItems 40176.7000 50.38%
removeUnusedSecondaryElements 28635.2000 35.91%
moveSecondaryFunctions 5998.9600 7.52%
writeModule_secondary 2611.0099 3.27%
writeModule_primary 935.7750 1.17%
exportImportCalledPrimaryFunctions 646.9860 0.81%
indirectReferencesToSecondaryFunctions 318.2980 0.40%
classifyFunctions 238.5780 0.30%
indirectCallsToSecondaryFunctions 139.1730 0.17%
setupTablePatching 44.1466 0.06%
thunkExportedSecondaryFunctions 3.9405 0.00%
initExportedPrimaryFuncs 0.6870 0.00%
---------------------------------------------------------------------------
Overall Total 79749.4539 100.00%
```
We can see time taken in `classifyFunctions`,
`indirectReferencesToSecondaryFunctions`, and
`indirectCallsToSecondaryFunctions` has reduced basically to nothing.
This is because now we can all functions only once in those functions,
where we used to scan the functions n times or similar.
Now `shareImportableItems` and `moveSecondaryFunctions` take up around
85% of the execution time. The reason `shareImportableItems` takes so
long is the reproducer has 90k globals.
```
Analysis of shareImportableItems:
Sub-Task Total Time (ms) Percentage
---------------------------------------------------------------------------
globals 41166.4904 98.35%
tables 535.5134 1.28%
tags 10.3355 0.02%
memories 7.5937 0.02%
exports 1.5482 0.00%
---------------------------------------------------------------------------
Total 41857.1000 100.00%
```
('exports' meaning processing existing exports)
We can probably improve this by selectively importing module items, as
already noted by the existing TODO.
`moveSecondaryFunctions` basically just runs RemoveUnusedModuleElements
on each module. We can also consider parallelizing
`moveSecondaryFunctions` by modules but not sure how much improvements
it can bring given that the pass is already parallized in function
granularity. But if we export only used items in `shareImportableItems`,
running this pass may become unnecessary after all.1 parent 959d522 commit 2089525
8 files changed
Lines changed: 419 additions & 378 deletions
File tree
- src
- ir
- tools/wasm-split
- test
- example
- lit/wasm-split
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
| 50 | + | |
| 51 | + | |
55 | 52 | | |
56 | 53 | | |
57 | 54 | | |
| |||
76 | 73 | | |
77 | 74 | | |
78 | 75 | | |
79 | | - | |
| 76 | + | |
80 | 77 | | |
81 | 78 | | |
82 | 79 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
329 | 329 | | |
330 | 330 | | |
331 | 331 | | |
332 | | - | |
| 332 | + | |
333 | 333 | | |
334 | 334 | | |
335 | 335 | | |
| |||
343 | 343 | | |
344 | 344 | | |
345 | 345 | | |
346 | | - | |
| 346 | + | |
347 | 347 | | |
348 | 348 | | |
349 | 349 | | |
| |||
389 | 389 | | |
390 | 390 | | |
391 | 391 | | |
392 | | - | |
393 | | - | |
394 | 392 | | |
395 | 393 | | |
396 | 394 | | |
397 | | - | |
| 395 | + | |
398 | 396 | | |
399 | 397 | | |
400 | 398 | | |
401 | 399 | | |
| 400 | + | |
402 | 401 | | |
403 | 402 | | |
| 403 | + | |
404 | 404 | | |
405 | 405 | | |
406 | 406 | | |
407 | 407 | | |
408 | 408 | | |
409 | 409 | | |
410 | 410 | | |
411 | | - | |
412 | | - | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
413 | 415 | | |
414 | 416 | | |
415 | 417 | | |
| |||
426 | 428 | | |
427 | 429 | | |
428 | 430 | | |
429 | | - | |
430 | 431 | | |
431 | 432 | | |
432 | 433 | | |
433 | 434 | | |
434 | 435 | | |
435 | 436 | | |
436 | 437 | | |
437 | | - | |
438 | | - | |
439 | | - | |
440 | | - | |
441 | | - | |
| 438 | + | |
442 | 439 | | |
443 | 440 | | |
444 | 441 | | |
445 | | - | |
446 | | - | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
447 | 446 | | |
448 | 447 | | |
449 | 448 | | |
450 | | - | |
451 | | - | |
452 | | - | |
453 | | - | |
| 449 | + | |
| 450 | + | |
454 | 451 | | |
455 | 452 | | |
456 | | - | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
457 | 462 | | |
458 | | - | |
459 | | - | |
460 | | - | |
461 | | - | |
462 | | - | |
463 | | - | |
464 | | - | |
465 | 463 | | |
466 | 464 | | |
467 | 465 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
51 | 51 | | |
52 | 52 | | |
53 | 53 | | |
54 | | - | |
| 54 | + | |
55 | 55 | | |
56 | | - | |
| 56 | + | |
| 57 | + | |
57 | 58 | | |
58 | 59 | | |
59 | 60 | | |
| |||
475 | 476 | | |
476 | 477 | | |
477 | 478 | | |
478 | | - | |
| 479 | + | |
479 | 480 | | |
480 | 481 | | |
481 | 482 | | |
482 | | - | |
| 483 | + | |
| 484 | + | |
483 | 485 | | |
484 | 486 | | |
485 | 487 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
10 | 14 | | |
11 | 15 | | |
12 | | - | |
13 | 16 | | |
14 | | - | |
15 | 17 | | |
16 | 18 | | |
17 | 19 | | |
| |||
20 | 22 | | |
21 | 23 | | |
22 | 24 | | |
23 | | - | |
| 25 | + | |
24 | 26 | | |
25 | | - | |
| 27 | + | |
26 | 28 | | |
27 | | - | |
| 29 | + | |
28 | 30 | | |
29 | | - | |
| 31 | + | |
30 | 32 | | |
31 | 33 | | |
32 | 34 | | |
| |||
36 | 38 | | |
37 | 39 | | |
38 | 40 | | |
39 | | - | |
| 41 | + | |
40 | 42 | | |
41 | 43 | | |
42 | 44 | | |
43 | 45 | | |
44 | | - | |
| 46 | + | |
45 | 47 | | |
46 | 48 | | |
47 | 49 | | |
| |||
71 | 73 | | |
72 | 74 | | |
73 | 75 | | |
74 | | - | |
| 76 | + | |
75 | 77 | | |
76 | | - | |
| 78 | + | |
77 | 79 | | |
78 | 80 | | |
79 | 81 | | |
| |||
92 | 94 | | |
93 | 95 | | |
94 | 96 | | |
95 | | - | |
| 97 | + | |
96 | 98 | | |
97 | 99 | | |
98 | 100 | | |
| |||
122 | 124 | | |
123 | 125 | | |
124 | 126 | | |
125 | | - | |
| 127 | + | |
126 | 128 | | |
127 | | - | |
| 129 | + | |
128 | 130 | | |
129 | 131 | | |
130 | 132 | | |
131 | | - | |
| 133 | + | |
132 | 134 | | |
133 | 135 | | |
134 | 136 | | |
| |||
167 | 169 | | |
168 | 170 | | |
169 | 171 | | |
170 | | - | |
| 172 | + | |
171 | 173 | | |
172 | | - | |
| 174 | + | |
173 | 175 | | |
174 | | - | |
| 176 | + | |
175 | 177 | | |
176 | | - | |
| 178 | + | |
177 | 179 | | |
178 | | - | |
| 180 | + | |
179 | 181 | | |
180 | | - | |
| 182 | + | |
181 | 183 | | |
182 | | - | |
183 | | - | |
184 | | - | |
185 | | - | |
186 | | - | |
187 | | - | |
188 | | - | |
189 | | - | |
190 | | - | |
191 | | - | |
192 | | - | |
| 184 | + | |
193 | 185 | | |
194 | 186 | | |
195 | 187 | | |
196 | | - | |
197 | | - | |
198 | | - | |
| 188 | + | |
199 | 189 | | |
200 | | - | |
201 | | - | |
| 190 | + | |
| 191 | + | |
202 | 192 | | |
203 | 193 | | |
204 | 194 | | |
205 | 195 | | |
206 | | - | |
207 | | - | |
208 | | - | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
209 | 199 | | |
210 | 200 | | |
211 | 201 | | |
212 | | - | |
213 | | - | |
214 | | - | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
215 | 205 | | |
216 | 206 | | |
0 commit comments