Implement native clang launcher - #27401
Conversation
198f9c8 to
c5fd0d6
Compare
c5fd0d6 to
3d058e6
Compare
470205f to
bac6b35
Compare
|
Here are some high level comments:
|
06b94ef to
9fec6c9
Compare
9fec6c9 to
b5f76dc
Compare
9ac1ab1 to
c94e316
Compare
|
I think once we land this and we start using it everywhere we can just remove the old shell/bat/ps1/pylauncher launchers. |
ac011fa to
bc1e09d
Compare
|
|
||
| namespace emscripten { | ||
|
|
||
| inline const std::unordered_set<std::string> LINK_ONLY_FLAGS = { |
There was a problem hiding this comment.
These can use std::string_view instead of std::string, and can be backed by the static C strings used in the initializers.
Gemini suggested even going further and using std::array instead of unordered_set, keeping them sorted, and use std::binary_search for the lookups. Not sure that's worth it though, it seems error-prone.
bc1e09d to
90c38f6
Compare
…ig file This ended up breaking emsdk in ways that would require more complexity to fix. See emscripten-core/emsdk#1752. This issue is that if emsdk is installed in the path contains spaces the NODE_JS path, for example, then needs to be quoted correctly in the config file. e.g. `NODE_JS = '"path with spaces/bin/node"'. Instead I think we should just revert this change. It turns out not to be needed for emscripten-core#27401 anyway since the LLVM_ROOT is never a list anyway.
This ended up breaking emsdk in ways that would require more complexity to fix. See emscripten-core/emsdk#1752. This issue is that if emsdk is installed in the path contains spaces the NODE_JS path, for example, then needs to be quoted correctly in the config file. e.g. `NODE_JS = '"path with spaces/bin/node"'. Instead I think we should just revert this change. It turns out not to be needed for #27401 anyway since the LLVM_ROOT is never a list anyway.
3fd0a72 to
b142bbf
Compare
|
I feel like this PR looking pretty good now. I've been back and forth with the AI agents reviewing the code mulitple times, with particular focus on any possible behaviour differences with the old pylauncher and/or run_python scripts. |
0b26dd3 to
5ed7d9f
Compare
Implement Phase 1 of the native launcher design document. Provides high-performance native C++ compiler drivers (emcc, em++) using C++20 that directly invoke Clang for compile steps (-c, -S, -E) and seamlessly fall back to python (emcc.py / em++.py) for link steps or complex post-processing options. Output executables are placed in ./bin using CMake and Ninja. Adds CI matrix testing for Linux, macOS, and Windows. These new binaries are currently 100% optional and the python versions can still be used without building them. Tested using `embuilder build libc --force` on Linux CI (compiling 1,075 files sequentially with `EMCC_CORES=1`, `EMCC_USE_NINJA=0`, and `EMCC_BATCH_BUILD=0`): | Metric | Before (Python Baseline) | After (Native Launcher) | Improvement | | :--- | :---: | :---: | :---: | | **Total Time** | **181.96 s** (3m 2s) | **64.82 s** (1m 5s) | **-117.14 s (64.4% faster)** | | **Time per File** | 169.3 ms | 60.3 ms | **-109.0 ms / invocation** | | **Speedup Factor** | 1.0x | **2.81x** | — | **Key Takeaways:** * **64.4% Reduction in Build Time:** Bypassing Python interpreter startup shaves off **~109.0 ms of overhead per compiler invocation** on Linux CI (a **2.81x speedup**). * **Windows Impact:** Because process spawning and `python.exe` startup carry significantly higher overhead on Windows than on Linux, the percentage speedup on Windows CI is expected to be even larger. See: emscripten-core#26453
5ed7d9f to
f60f9dd
Compare
Implement Phase 1 of the native launcher design document. Provides
high-performance native C++ compiler drivers (
emcc,em++) using C++20that directly invoke Clang for compile steps (-c, -S, -E) and seamlessly
fall back to python (
emcc.py/em++.py) for link steps or complexpost-processing options.
This tool completely replaces both the windows-specific
pylauncherandthe shell/bat entry points created by
create_entry_points.pyThese new binaries are currently optional since
create_entry_points.pycan still be used instead.
Tested using
embuilder build libc --force(compiling 1,075files sequentially with
EMCC_CORES=1,EMCC_USE_NINJA=0, andEMCC_BATCH_BUILD=0). Note that this is likely an outlier sinceits perfectly suited to demonstrating to speedup from this change (i.e.
lots of small source files and no linking).
See: #26453