[24.04_linux-nvidia-6.17-next] linux-nvidia-6.17: SMT-aware asymmetric CPU capacity idle selection#395
Conversation
PR Validation ReportPatchscan ✅ No Missing FixesAll cherry-picked commits checked — no missing upstream fixes found. PR Lint ❌ Errors foundDetailsChecking 4 commits...
Cherry-pick digest:
E: 9a87c418e16e ("NVIDIA: SAUCE: sched/fair: Attach sched_"): diff MISMATCH with lore patch (add [Author: reason] annotation if intentional)
┌──────────────┬───────────────────────────────────────────────┬────────────┬─────────┬───────────────────────────┐
│ Local │ Referenced upstream / Patch subject │ Patch-ID │ Subject │ SoB chain │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ b3bf56e2a412 │ [SAUCE] sched/fair: add sis_util support to s │ N/A │ N/A │ arighi, nayak, arighi │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ 842b1195ebce │ [SAUCE] sched/fair: reject misfit pulls onto │ N/A │ N/A │ arighi, arighi │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ 20f85dc0e99e │ [SAUCE] sched/fair: prefer fully-idle smt cor │ N/A │ N/A │ arighi, arighi │
├──────────────┼───────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ 9a87c418e16e │ sched/fair: attach sched_domain_shared to sd_ │ MISMATCH │ found │ ok, backporter: arighi │
└──────────────┴───────────────────────────────────────────────┴────────────┴─────────┴───────────────────────────┘
Lint: all checks passed.
|
arighi
changed the title
arighi
changed the title
|
@arighi |
|
The upstream v4 series has 6 patches but this PR only carries 4 (patches 2/6–5/6). Are the other two patches in the series -- |
arighi
force-pushed
the
linux-nvidia-6.17-sched
branch
from
c6377a1 to
4ef9f0b
Compare
|
@clsotog oops I forgot to update the subject of the patch, and yes, it is PATCH 2/6. Fixed now. Thanks! |
|
@jamieNguyenNVIDIA right, I should have mentioned in the PR description, sorry. The first patch and the last one are not needed. Updated the PR description. Thanks! |
|
Be sure to send this to 26.04_linux-nvidia and 26.04_linux-nvidia-bos. Thanks! |
| per_cpu(sd_llc_size, cpu) = size; | ||
| per_cpu(sd_llc_id, cpu) = id; | ||
|
|
||
| /* TODO: Rename sd_llc_shared to fit the new role. */ |
There was a problem hiding this comment.
Do we still have a TODO here?
There was a problem hiding this comment.
I was a bit conflicted on this one, because the patch that renames sd_llc_shared to sd_balance_shared includes other changes that shouldn't be backported to 6.17, so I left that comment there, but I guess we can remove it and just keep sd_llc_shared.
There was a problem hiding this comment.
@clsotog thinking more about it, I just removed the TODO, considering that we're not backporting the patch that renames the struct, the TODO doesn't really add any useful information and it's just confusing. PR updated.
|
@arighi Couple of questions... 4ef9f0b NVIDIA: SAUCE: sched/fair: Add SIS_UTIL support to select_idle_capacity() Nit: These are not clean picks from LKML (looks like delta is just in the comments). Are these from on an older version of the series? 467bd61 NVIDIA: SAUCE: sched/fair: Attach sched_domain_shared to sd_asym_cpucapacity Similar: The commit message differs from the LKML version. |
arighi
force-pushed
the
linux-nvidia-6.17-sched
branch
from
4ef9f0b to
9ec2612
Compare
|
@nvmochs ah yes, these patches are based on an older series, I just updated the code without updating the comments. Better to keep them aligned. I've updated the PR, now they should match with the upstream patches. Thanks! |
…apacity
On asymmetric CPU capacity systems, the wakeup path uses
select_idle_capacity(), which scans the span of sd_asym_cpucapacity
rather than sd_llc.
The has_idle_cores hint however lives on sd_llc->shared, so the
wakeup-time read of has_idle_cores operates on an LLC-scoped blob while
the actual scan/decision spans the wider asym domain; nr_busy_cpus also
lives in the same shared sched_domain data, but it's never used in the
asym CPU capacity scenario.
Therefore, move the sched_domain_shared object to sd_asym_cpucapacity
whenever the CPU has a SD_ASYM_CPUCAPACITY_FULL ancestor and that
ancestor is non-overlapping (i.e., not built from SD_NUMA). In that case
the scope of has_idle_cores matches the scope of the wakeup scan.
Fall back to attaching the shared object to sd_llc in three cases:
1) plain symmetric systems (no SD_ASYM_CPUCAPACITY_FULL anywhere);
2) CPUs in an exclusive cpuset that carves out a symmetric capacity
island: has_asym is system-wide but those CPUs have no
SD_ASYM_CPUCAPACITY_FULL ancestor in their hierarchy and follow
the symmetric LLC path in select_idle_sibling();
3) exotic topologies where SD_ASYM_CPUCAPACITY_FULL lands on an
SD_NUMA-built domain. init_sched_domain_shared() keys the shared
blob off cpumask_first(span), which on overlapping NUMA domains
would alias unrelated spans onto the same blob. Keep the shared
object on the LLC there; select_idle_capacity() gracefully skips
the has_idle_cores preference when sd->shared is NULL.
While at it, also rename the per-CPU sd_llc_shared to sd_balance_shared,
as it is no longer strictly tied to the LLC.
Co-developed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: K Prateek Nayak <kprateek.nayak@amd.com>
(backported from https://lore.kernel.org/all/20260428051720.3180182-1-arighi@nvidia.com)
[ arighi:
- backport full logic to attach sd->shared in build_sched_domains()
- do not rename sd_llc_shared to reduce the risk of conflicts ]
Signed-off-by: Andrea Righi <arighi@nvidia.com>
…ty idle selection On systems with asymmetric CPU capacity (e.g., ACPI/CPPC reporting different per-core frequencies), the wakeup path uses select_idle_capacity() and prioritizes idle CPUs with higher capacity for better task placement. However, when those CPUs belong to SMT cores, their effective capacity can be much lower than the nominal capacity when the sibling thread is busy: SMT siblings compete for shared resources, so a "high capacity" CPU that is idle but whose sibling is busy does not deliver its full capacity. This effective capacity reduction cannot be modeled by the static capacity value alone. Introduce SMT awareness in the asym-capacity idle selection policy: when SMT is active, always prefer fully-idle SMT cores over partially-idle ones. Prioritizing fully-idle SMT cores yields better task placement because the effective capacity of partially-idle SMT cores is reduced; always preferring them when available leads to more accurate capacity usage on task wakeup. On an SMT system with asymmetric CPU capacities, SMT-aware idle selection has been shown to improve throughput by around 15-18% for CPU-bound workloads, running an amount of tasks equal to the amount of SMT cores. Cc: Vincent Guittot <vincent.guittot@linaro.org> Cc: Dietmar Eggemann <dietmar.eggemann@arm.com> Cc: Christian Loehle <christian.loehle@arm.com> Cc: Koba Ko <kobak@nvidia.com> Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com> Reported-by: Felix Abecassis <fabecassis@nvidia.com> Signed-off-by: Andrea Righi <arighi@nvidia.com> (cherry picked from https://lore.kernel.org/all/20260428051720.3180182-1-arighi@nvidia.com) Signed-off-by: Andrea Righi <arighi@nvidia.com>
… on asym-capacity When SD_ASYM_CPUCAPACITY load balancing considers pulling a misfit task, capacity_of(dst_cpu) can overstate available compute if the SMT sibling is busy: the core does not deliver its full nominal capacity. If SMT is active and dst_cpu is not on a fully idle core, skip this destination so we do not migrate a misfit expecting a capacity upgrade we cannot actually provide. Cc: Vincent Guittot <vincent.guittot@linaro.org> Cc: Dietmar Eggemann <dietmar.eggemann@arm.com> Cc: Christian Loehle <christian.loehle@arm.com> Cc: Koba Ko <kobak@nvidia.com> Cc: K Prateek Nayak <kprateek.nayak@amd.com> Reported-by: Felix Abecassis <fabecassis@nvidia.com> Signed-off-by: Andrea Righi <arighi@nvidia.com> (cherry picked from https://lore.kernel.org/all/20260428051720.3180182-1-arighi@nvidia.com) Signed-off-by: Andrea Righi <arighi@nvidia.com>
…ty() Add to select_idle_capacity() the same SIS_UTIL-controlled idle-scan mechanism, already used by select_idle_cpu(): when sched_feat(SIS_UTIL) is enabled and the LLC domain has sched_domain_shared data, derive the per-attempt scan limit from sd->shared->nr_idle_scan. That bounds the walk on large LLCs and allows an early return once the scan limit is reached, if we already picked a sufficiently strong idle-core candidate (best_fits == ASYM_IDLE_CORE_UCLAMP_MISFIT). Co-developed-by: Andrea Righi <arighi@nvidia.com> Signed-off-by: Andrea Righi <arighi@nvidia.com> Signed-off-by: K Prateek Nayak <kprateek.nayak@amd.com> (cherry picked from https://lore.kernel.org/all/20260428051720.3180182-1-arighi@nvidia.com) Signed-off-by: Andrea Righi <arighi@nvidia.com>
arighi
force-pushed
the
linux-nvidia-6.17-sched
branch
from
9ec2612 to
b3bf56e
Compare
|
No further issues from me.
FYI, I went ahead and created a Launchpad (required for merging) at added the link in the PR description. Also, as Jamie mentioned, we'll want this backported to 26.04 7.0-LTS and 7.0-bos. However, I recommend we wait until we can pick from -next to avoid carrying this as SAUCE in those kernels. It sounds like the series is getting close to being accepted, so this shouldn't be too far in the future. I'll open Jiras to track getting this into those kernels. |
clsotog
left a comment
clsotog
left a comment
There was a problem hiding this comment.
Acked-by: Carol L Soto <csoto@nvidia.com>
|
Merged, closing PR. |
5 participants
On Vera Rubin, the firmware exposes CPUs with different capacities through ACPI/CPPC. Unlike Grace systems, Vera Rubin also supports SMT. As a result, the Linux scheduler enables the asymmetric CPU capacity idle selection policy, but the current implementation is not SMT-aware. This can lead to suboptimal task placement, where tasks are scheduled on both SMT siblings of the same core even when fully idle SMT cores are available elsewhere in the system.
In CPU-intensive workloads, this behavior can significantly reduce performance, with slowdowns of up to 2x observed in certain CPU-intensive workloads.
This series is a backport of the upstream patch series available at:
https://lore.kernel.org/all/20260428051720.3180182-1-arighi@nvidia.com
NOTE: the original series includes additional patches that are not needed in linux-nvidia-6.17:
71fedc41c23b ("sched/fair: Switch to rcu_dereference_all()")and it's not worth backporting it,The series is currently under review on the mailing list, but consensus has been reached with the scheduler maintainers and the changes are expected to be merged for v7.2.
Given the potential impact on Vera Rubin performance, it seems reasonable to backport and apply these patches to the linux-nvidia kernel and carry them as NVIDIA SAUCE for now, until the upstream solution becomes available.
Patch series has been validated both on Vera and Grace running DCPerf Mediawiki and benchblas (NVBLAS).
LP: https://bugs.launchpad.net/ubuntu/+source/linux-nvidia-bos/+bug/2150671