Skip to content

Commit 85b353e

Browse files
LpZ-squallclaude
andcommitted
waf/backend: limit parallel link tasks with MAX_LINK_TASKS.
Linking is memory-intensive, so on machines with many CPUs, running too many link tasks in parallel can exhaust memory and disk I/O. Use waf's TaskSemaphore to cap link task parallelism to 8 by default, configurable via the MAX_LINK_TASKS environment variable. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Change-Id: I401dbbc18e2b76e7796aec77b4b7f6fe495e0131 Priv-Id: ae987beb7863683fd5cd5c7acbcc4152fe84d027
1 parent b8025f1 commit 85b353e

2 files changed

Lines changed: 16 additions & 1 deletion

File tree

README.adoc

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -266,6 +266,14 @@ The following environment variables can be used at the configuration phase:
266266
If the variable is not set, only the release and dev dependencies are
267267
installed.
268268

269+
The following environment variables can be used at the build phase:
270+
271+
`MAX_LINK_TASKS` (integer)::
272+
Limit the number of link tasks that can run in parallel. Linking is
273+
memory-intensive, so on machines with many CPUs, running too many link
274+
tasks in parallel can exhaust memory and disk I/O.
275+
Defaults to 8 if not set.
276+
269277
==== Compiling with gcc
270278

271279
An important part of the lib-common uses the

build/waftools/backend.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@
4949
from waflib.Configure import ConfigurationContext
5050
from waflib.Node import Node
5151
from waflib.Options import OptionsContext
52-
from waflib.Task import Task
52+
from waflib.Task import Task, TaskSemaphore
5353
from waflib.Tools import (
5454
c as c_tool,
5555
c_preproc,
@@ -2055,6 +2055,13 @@ def build(ctx: BuildContext) -> None:
20552055
ctx.env.GEN_FILES = set()
20562056
ctx.env.CHECKED_FILES = set()
20572057

2058+
# Limit the number of parallel link tasks. Linking is memory-intensive,
2059+
# so on machines with many CPUs, running too many link tasks in parallel
2060+
# can exhaust memory and disk I/O.
2061+
max_link_tasks = int(os.environ.get('MAX_LINK_TASKS', '8'))
2062+
Logs.info(f'Waf: Limiting parallel link tasks to {max_link_tasks}')
2063+
ccroot.link_task.semaphore = TaskSemaphore(max_link_tasks)
2064+
20582065
register_get_cwd()
20592066

20602067
# iopc options

0 commit comments

Comments
 (0)