Skip to content

Commit 834f278

Browse files
committed
feat(wasm): scale pthread pool and GDAL threads to device CPU count
Multi-thread WASM builds now size the pthread pool to navigator.hardwareConcurrency at module load instead of a fixed 4. PTHREAD_POOL_SIZE_STRICT=2 makes pthread overflow abort cleanly instead of risking main-thread deadlock from dynamic pool growth. GDAL package config follows suit: GDAL_NUM_THREADS now uses 'ALL_CPUS' on mt builds (was '4'), letting GDAL scale to the device's actual CPU count. Updates docs/api/performance.md and website/docs/api/configuration/performance.md: defaults table, tunable section, and the "common mistakes" item.
1 parent 45fc1ed commit 834f278

4 files changed

Lines changed: 49 additions & 28 deletions

File tree

cppjs-core/cpp.js/src/actions/buildWasm.js

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,8 @@ export default async function buildWasm(target, options = {}) {
3030

3131
if (target.runtime === 'mt' && !emccFlags.includes('-pthread')) {
3232
emccFlags.push('-pthread');
33-
emccFlags.push('-sPTHREAD_POOL_SIZE=4');
33+
emccFlags.push('-sPTHREAD_POOL_SIZE=navigator.hardwareConcurrency');
34+
emccFlags.push('-sPTHREAD_POOL_SIZE_STRICT=2');
3435
}
3536

3637
if (target.platform === 'wasm') {

cppjs-packages/cppjs-package-gdal/cppjs-package-gdal-wasm/cppjs.config.js

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ export default {
5151
DXF_FEATURE_LIMIT_PER_BLOCK: '-1',
5252
GDAL_ENABLE_DEPRECATED_DRIVER_GTM: 'YES',
5353
CPL_LOG_ERRORS: 'ON',
54-
GDAL_NUM_THREADS: (state, target) => (target.runtime === 'st' ? '0' : '4'),
54+
GDAL_NUM_THREADS: (state, target) => (target.runtime === 'st' ? '0' : 'ALL_CPUS'),
5555
}
5656
}
5757
}

docs/api/performance.md

Lines changed: 23 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,8 @@ The rule: **if your build runs and your app works, the defaults are fine**. Only
1414
| `-O0` | (debug) | No optimization | ✅ Already debug — fine |
1515
| `-msimd128` | wasm | SIMD128 instruction set | 🔒 Already optimal |
1616
| `-sMEMORY64=1` | wasm64 only | 64-bit memory | 🔒 Set by target.arch, not flag override |
17-
| `-pthread` + `-sPTHREAD_POOL_SIZE=4` | mt only | Thread pool |`PTHREAD_POOL_SIZE` is tunable (see below) |
17+
| `-pthread` + `-sPTHREAD_POOL_SIZE=navigator.hardwareConcurrency` | mt only | Thread pool sized to device CPU at module load |`PTHREAD_POOL_SIZE` is tunable (see below) |
18+
| `-sPTHREAD_POOL_SIZE_STRICT=2` | mt only | Abort if more pthreads requested than pool (no dynamic growth) | ⚠️ Drop to `1` (warn + grow) or `0` (silent grow) only if your code spawns unbounded threads |
1819
| `-lembind` | always | Embind binding lib | 🔒 Required |
1920
| `-Wl,--whole-archive` | always | Link all objects | 🔒 Required for static lib symbol retention |
2021
| `-fwasm-exceptions` | always | C++ exceptions via Wasm EH | 🔒 Required for proper `throw` semantics |
@@ -86,21 +87,30 @@ target: { arch: 'wasm64' }
8687

8788
Don't try to push wasm32 past 4GB — Wasm spec doesn't allow it.
8889

89-
### `PTHREAD_POOL_SIZE` (default: 4)
90+
### `PTHREAD_POOL_SIZE` (default: `navigator.hardwareConcurrency`)
9091

91-
Default 4 worker threads in the pool. Match to your workload:
92+
The pool is sized to the device's CPU count at module load — `navigator.hardwareConcurrency` is evaluated as a JS expression by Emscripten at runtime, not baked in at build time. Most apps need no override.
9293

93-
- Image / video / geo / crypto with `runtime: 'mt'`: bump to `navigator.hardwareConcurrency` worth.
94-
- Background tasks where you want main thread responsive: leave at 4 or lower.
94+
Paired with `-sPTHREAD_POOL_SIZE_STRICT=2`: if your code requests more pthreads than the pool, the runtime **aborts**. No dynamic growth, no main-thread deadlock risk. If your thread count is bounded by `hardwareConcurrency` you're fine; otherwise either bump the pool or relax strictness.
9595

96-
```js
97-
targetSpecs: [{
98-
platform: 'wasm', runtime: 'mt',
99-
specs: { emccFlags: ['-sPTHREAD_POOL_SIZE=8'] },
100-
}]
101-
```
96+
Override only when:
97+
98+
- **Background tasks where you want the main thread responsive** — cap below CPU count:
99+
```js
100+
targetSpecs: [{
101+
platform: 'wasm', runtime: 'mt',
102+
specs: { emccFlags: ['-sPTHREAD_POOL_SIZE=2'] },
103+
}]
104+
```
105+
- **Workloads that spawn more threads than CPU count** — bump pool size, or relax strict mode:
106+
```js
107+
targetSpecs: [{
108+
platform: 'wasm', runtime: 'mt',
109+
specs: { emccFlags: ['-sPTHREAD_POOL_SIZE=16', '-sPTHREAD_POOL_SIZE_STRICT=1'] },
110+
}]
111+
```
102112

103-
Spawning more than `hardwareConcurrency` doesn't help — context-switching costs dominate.
113+
Spawning more than `hardwareConcurrency` doesn't speed things up — context-switching costs dominate.
104114

105115
### `RESERVED_FUNCTION_POINTERS` (default: 200)
106116

@@ -197,7 +207,7 @@ Your C++ might not throw, but std::vector / std::string / std::map can. Removing
197207

198208
### "I'll bump `PTHREAD_POOL_SIZE` to 32 for max parallelism"
199209

200-
Going past `hardwareConcurrency` adds context-switching overhead. Most users have 4-8 cores; pool size 4 is the right default. Bump only when you've measured a benefit.
210+
Going past `hardwareConcurrency` adds context-switching overhead. The default already matches the device CPU count at runtime — bumping it past that doesn't help. Bump only when your code spawns more concurrent threads than the device has cores AND you've measured a benefit (you'll also need `-sPTHREAD_POOL_SIZE_STRICT=1` so the extra threads don't trip the strict abort).
201211

202212
## Profiling
203213

website/docs/api/configuration/performance.md

Lines changed: 23 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,8 @@ The rule: **if your build runs and your app works, the defaults are fine**. Only
1414
| `-O0` | (debug) | No optimization | ✅ Already debug — fine |
1515
| `-msimd128` | wasm | SIMD128 instruction set | 🔒 Already optimal |
1616
| `-sMEMORY64=1` | wasm64 only | 64-bit memory | 🔒 Set by `target.arch`, not flag override |
17-
| `-pthread` + `-sPTHREAD_POOL_SIZE=4` | mt only | Thread pool |`PTHREAD_POOL_SIZE` is tunable |
17+
| `-pthread` + `-sPTHREAD_POOL_SIZE=navigator.hardwareConcurrency` | mt only | Thread pool sized to device CPU at module load |`PTHREAD_POOL_SIZE` is tunable (see below) |
18+
| `-sPTHREAD_POOL_SIZE_STRICT=2` | mt only | Abort if more pthreads requested than pool (no dynamic growth) | ⚠️ Drop to `1` (warn + grow) or `0` (silent grow) only if your code spawns unbounded threads |
1819
| `-lembind` | always | Embind binding lib | 🔒 Required |
1920
| `-Wl,--whole-archive` | always | Link all objects | 🔒 Required for static lib symbol retention |
2021
| `-fwasm-exceptions` | always | C++ exceptions via Wasm EH | 🔒 Required for proper `throw` semantics |
@@ -86,21 +87,30 @@ target: { arch: 'wasm64' }
8687

8788
Don't try to push wasm32 past 4GB — Wasm spec doesn't allow it.
8889

89-
### `PTHREAD_POOL_SIZE` (default: 4)
90+
### `PTHREAD_POOL_SIZE` (default: `navigator.hardwareConcurrency`)
9091

91-
Default 4 worker threads in the pool. Match to your workload:
92+
The pool is sized to the device's CPU count at module load — `navigator.hardwareConcurrency` is evaluated as a JS expression by Emscripten at runtime, not baked in at build time. Most apps need no override.
9293

93-
- Image / video / geo / crypto with `runtime: 'mt'`: bump to `navigator.hardwareConcurrency` worth.
94-
- Background tasks where you want main thread responsive: leave at 4 or lower.
94+
Paired with `-sPTHREAD_POOL_SIZE_STRICT=2`: if your code requests more pthreads than the pool, the runtime **aborts**. No dynamic growth, no main-thread deadlock risk. If your thread count is bounded by `hardwareConcurrency` you're fine; otherwise either bump the pool or relax strictness.
9595

96-
```js
97-
targetSpecs: [{
98-
platform: 'wasm', runtime: 'mt',
99-
specs: { emccFlags: ['-sPTHREAD_POOL_SIZE=8'] },
100-
}]
101-
```
96+
Override only when:
97+
98+
- **Background tasks where you want the main thread responsive** — cap below CPU count:
99+
```js
100+
targetSpecs: [{
101+
platform: 'wasm', runtime: 'mt',
102+
specs: { emccFlags: ['-sPTHREAD_POOL_SIZE=2'] },
103+
}]
104+
```
105+
- **Workloads that spawn more threads than CPU count** — bump pool size, or relax strict mode:
106+
```js
107+
targetSpecs: [{
108+
platform: 'wasm', runtime: 'mt',
109+
specs: { emccFlags: ['-sPTHREAD_POOL_SIZE=16', '-sPTHREAD_POOL_SIZE_STRICT=1'] },
110+
}]
111+
```
102112

103-
Spawning more than `hardwareConcurrency` doesn't help — context-switching costs dominate.
113+
Spawning more than `hardwareConcurrency` doesn't speed things up — context-switching costs dominate.
104114

105115
### `RESERVED_FUNCTION_POINTERS` (default: 200)
106116

@@ -197,7 +207,7 @@ Your C++ might not throw, but `std::vector` / `std::string` / `std::map` can. Re
197207

198208
### "I'll bump `PTHREAD_POOL_SIZE` to 32 for max parallelism"
199209

200-
Going past `hardwareConcurrency` adds context-switching overhead. Most users have 4-8 cores; pool size 4 is the right default. Bump only when you've measured a benefit.
210+
Going past `hardwareConcurrency` adds context-switching overhead. The default already matches the device CPU count at runtime — bumping it past that doesn't help. Bump only when your code spawns more concurrent threads than the device has cores AND you've measured a benefit (you'll also need `-sPTHREAD_POOL_SIZE_STRICT=1` so the extra threads don't trip the strict abort).
201211

202212
## Profiling
203213

0 commit comments

Comments
 (0)