Skip to content

[infra] Reduce default build parallelism #69

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
vikuznetsov-nvidia wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[infra] Reduce default build parallelism #69

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions build_release.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# --build_docs Build documentation
# --build_type=TYPE Set CMake build type (Debug|Release[default]|RelWithDebInfo|MinSizeRel)
# --cuda_arch=ARCH Set CUDA architecture target (e.g. 87 for Orin Nano, default: all)
# --jobs=N Set number of parallel jobs (default: 8)
# --jobs=N Set parallel jobs (default: ceil(nproc/2))
#
# Environment variables (optional):
# CUVSLAM_SRC_DIR Source directory (default: /cuvslam/src)
Expand Down Expand Up @@ -46,7 +46,8 @@ USE_RERUN=OFF
SRC=/cuvslam/src
DST=/cuvslam/build

MAKE_JOBS=$(nproc 2>/dev/null || echo 8)
nproc_n=$(nproc 2>/dev/null || echo 8)
MAKE_JOBS=$(( (nproc_n + 1) / 2 ))

if [ ${CUVSLAM_SRC_DIR+x} ]; then
echo "Using environment variable CUVSLAM_SRC_DIR='$CUVSLAM_SRC_DIR'."
Expand Down
11 changes: 6 additions & 5 deletions scripts/build_cuvslam_in_docker.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ if [ "$#" -lt 1 ] || [ "$#" -gt 2 ]; then
echo " EXTRA_CMAKE_ARGS Additional CMake arguments"
echo " CUVSLAM_REQUIRE_CLEAN_SOURCE Fail if tracked source files differ from HEAD"
echo " BUILD_JOBS Parallel build jobs, passed as --jobs to build_release.sh"
echo " (default: min(nproc, MemAvailable/4GB) to avoid OOM)"
echo " (default: min(ceil(nproc/2), MemAvailable/4GB))"
exit 1
fi

Expand All @@ -37,19 +37,20 @@ DOCKER_USER="--user $(id -u):$(id -g) --group-add video -e HOME=/tmp"
TTY_FLAG=""
[ -t 0 ] && TTY_FLAG="-it"

# Cap parallelism so each compile job has ~4GB of available memory, avoiding OOM
# on memory-constrained runners (e.g. Jetson Thor). Override by setting BUILD_JOBS.
# Cap parallelism at half the available CPUs to leave power and SMT headroom,
# and at ~4GB of available memory per compile job. Override by setting BUILD_JOBS.
if [ -z "${BUILD_JOBS:-}" ]; then
nproc_n=$(nproc)
cpu_cap=$(( (nproc_n + 1) / 2 ))
mem_cap=$(awk '/MemAvailable/ { print int($2/1024/1024/4) }' /proc/meminfo)
if [ "$mem_cap" -lt 1 ]; then
mem_cap=1
fi
BUILD_JOBS=$nproc_n
BUILD_JOBS=$cpu_cap
if [ "$mem_cap" -lt "$BUILD_JOBS" ]; then
BUILD_JOBS=$mem_cap
fi
echo "Build parallelism: --jobs=$BUILD_JOBS (nproc=$nproc_n, mem-capped jobs=${mem_cap} at 4GB/job)"
echo "Build parallelism: --jobs=$BUILD_JOBS (nproc=$nproc_n, CPU cap=$cpu_cap, memory cap=$mem_cap at 4GB/job)"
fi

JOBS_ARG=()
Expand Down
Loading