docs: parallelize man page generation with make -j#10775
Merged
maliberty merged 1 commit intoJun 29, 2026
Merged
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request updates bazel/man_pages.bzl to split the man page generation into a serial preprocessing phase and a parallelized phase. The feedback correctly identifies that using nproc directly will cause build failures on macOS and suggests a robust fallback using sysctl.
Member
Author
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8be0cfdb4d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The man_pages Bazel action invoked 'make bazel-manpages' serially, running ~10,000 pandoc/nroff invocations one at a time (~177s on a 64-core host). Split the single make call into two phases: 'preprocess' (serial) generates the md/man*/*.md sources, then 'cat web' fan out pandoc/nroff in parallel with -j. They cannot share one -j invocation because cat/web read the md files that preprocess produces and there is no dependency edge forcing preprocess first. Running 'cat web' as a second invocation also re-parses the Makefile so its $(wildcard md/man*/*.md) picks up the freshly generated sources. Job count uses nproc with a sysctl fallback (then 4) so the action also works on macOS, where nproc is not available by default. A resource_set reserves the host CPUs for this action so the parallel make does not oversubscribe other concurrently scheduled Bazel actions; Bazel clamps the request to the cores actually available. Regenerating the full doc set drops from ~177s to ~6s (~30x). Signed-off-by: Matt Liberty <mliberty@precisioninno.com>
openroad-ci
force-pushed
the
man-pages-parallel-gen
branch
from
8be0cfd to
2644a56
Compare
maliberty
merged commit 5fcc1e8
into
The-OpenROAD-Project:master
16 of 17 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
man_pagesBazel action invokedmake bazel-manpagesserially, running ~10,000 pandoc/nroff invocations one at a time. On a 64-core host this took ~177s and was the long pole of//:install.Split the single
makecall into two phases:preprocess(serial) — generates themd/man*/*.mdsources.cat web(parallel,-j$(nproc)) — the pandoc/nroff fan-out over ~3,351 pages.They cannot share one
-jinvocation:cat/webread the md filespreprocessproduces, and a parallel build has no dependency edge forcingpreprocessto finish first. Runningcat webas a second invocation also re-parses the Makefile so its$(wildcard md/man*/*.md)picks up the freshly generated sources.Results
Regenerating the full doc set (3113 man3 + 237 man2 + 1 man1, cat + html) from a cold action cache:
-j64)~30× faster. Output set is byte-for-byte identical in page counts (cat3=3113, html3=3113, cat2=237, cat1=1).