fix(build): cap default build parallelism by available memory to prevent OOM

[mc-nv]

^{CI (internal): [#57283463](https://gitlab-master.nvidia.com/dl/dgx/tritonserver/-/pipelines/57283463)}

What does the PR do?

Caps the default build.py parallelism so fully-uncached builds no longer OOM the
GitLab runner hosts.

Previously the default was multiprocessing.cpu_count() * 2, which has two problems for
Triton's memory-heavy C++/CUDA compilation (including the nested ONNX Runtime build):

• multiprocessing.cpu_count() reports the host's logical core count — it ignores CPU
  affinity and cgroup quotas, so a constrained build on a large host could spawn far more
  jobs than intended.
• Template-dense translation units routinely peak at 1–2+ GB RAM per compiler process, so
  2 * cores oversubscribes memory on swap-less builders and the OOM killer terminates
  runner/system daemons instead of the build, leaving jobs stuck or orphaned.

The default is now computed by default_build_parallel(), which:

• counts usable cores via os.sched_getaffinity(0) (respects pinning/cgroups), falling
  back to multiprocessing.cpu_count() on non-Linux platforms;
• caps -j so each compile job has ~2 GB of RAM headroom, based on MemAvailable from
  /proc/meminfo;
• logs when it throttles parallelism below the usable core count.

Passing -j / --build-parallel explicitly is unaffected — only the default changes.

Checklist

• PR title reflects the change and is of format <commit_type>: <Title>
• Changes are described in the pull request.
• Related issues are referenced.
• Populated github labels field
• Added test plan and verified test passes.
• Verified that the PR passes existing CI.
• Verified copyright is correct on all changed files.
• Added succinct git squash message before merging ref.
• All template sections are filled out.
• Optional: Additional screenshots for behavior/output changes with before/after.

Commit Type:

• fix

Related PRs:

triton-inference-server/onnxruntime_backend#348

Where should the reviewer start?

build.py — the new usable_cpu_count(), available_memory_gb(), and
default_build_parallel() helpers, and the default_build_parallel() call at the
FLAGS.build_parallel is None default site.

Test plan:

• Simulated the parallelism calculation across representative host profiles:

  • 128 cores / 32 GB → 16 jobs (was 256)
  • 8 cores / 64 GB → 8 jobs
  • 16 cores / 1 GB → 1 job
  • macOS / no /proc/meminfo → CPU-only fallback (usable_cpu_count())

• python3 -c "import ast; ast.parse(open('build.py').read())" passes.

• Validate on the affected builder / arm-builder jobs that a fully uncached build
  no longer OOMs the runner.

• CI Pipeline ID: 57283463

Caveats:

• The ~2 GB/job headroom is a conservative constant; the heaviest TensorRT/ONNX TUs can
  spike higher. Tunable in-source if needed.
• In the Docker (--enable-all) path the default is computed on the host and baked into
  the generated build scripts; if a build container is constrained with --memory, the
  host's MemAvailable won't reflect that cap. Can be extended to read the cgroup
  memory.max inside the build stage if CI constrains container memory.
• os.sched_getaffinity is Linux-only (hence the multiprocessing.cpu_count() fallback).

Background

GitLab build jobs were failing with logs abruptly stopping because runner hosts OOM'd
during fully uncached builds — unbounded parallelism in the Tritonserver build (including
the nested ONNX Runtime build) on swap-less builders.

Related Issues:

• Resolves: TRI-1550

[whoisj]

neat trick!