Skip to content

Commit 864359d

Browse files
rgushchinAvenger-285714
authored andcommitted
percpu: scoped objcg protection
mainline inclusion from mainline-v6.7-rc1 category: performance commit c63b835 upstream. Similar to slab and kmem, switch to a scope-based protection of the objcg pointer to avoid. Link: https://lkml.kernel.org/r/20231019225346.1822282-6-roman.gushchin@linux.dev Signed-off-by: Roman Gushchin (Cruise) <roman.gushchin@linux.dev> Tested-by: Naresh Kamboju <naresh.kamboju@linaro.org> Acked-by: Shakeel Butt <shakeelb@google.com> Reviewed-by: Vlastimil Babka <vbabka@suse.cz> Cc: David Rientjes <rientjes@google.com> Cc: Dennis Zhou <dennis@kernel.org> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Muchun Song <muchun.song@linux.dev> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> (cherry picked from commit c63b835) Signed-off-by: Wentao Guan <guanwentao@uniontech.com>
1 parent b382bbb commit 864359d

1 file changed

Lines changed: 3 additions & 5 deletions

File tree

mm/percpu.c

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1628,14 +1628,12 @@ static bool pcpu_memcg_pre_alloc_hook(size_t size, gfp_t gfp,
16281628
if (!memcg_kmem_online() || !(gfp & __GFP_ACCOUNT))
16291629
return true;
16301630

1631-
objcg = get_obj_cgroup_from_current();
1631+
objcg = current_obj_cgroup();
16321632
if (!objcg)
16331633
return true;
16341634

1635-
if (obj_cgroup_charge(objcg, gfp, pcpu_obj_full_size(size))) {
1636-
obj_cgroup_put(objcg);
1635+
if (obj_cgroup_charge(objcg, gfp, pcpu_obj_full_size(size)))
16371636
return false;
1638-
}
16391637

16401638
*objcgp = objcg;
16411639
return true;
@@ -1649,6 +1647,7 @@ static void pcpu_memcg_post_alloc_hook(struct obj_cgroup *objcg,
16491647
return;
16501648

16511649
if (likely(chunk && chunk->obj_cgroups)) {
1650+
obj_cgroup_get(objcg);
16521651
chunk->obj_cgroups[off >> PCPU_MIN_ALLOC_SHIFT] = objcg;
16531652

16541653
rcu_read_lock();
@@ -1657,7 +1656,6 @@ static void pcpu_memcg_post_alloc_hook(struct obj_cgroup *objcg,
16571656
rcu_read_unlock();
16581657
} else {
16591658
obj_cgroup_uncharge(objcg, pcpu_obj_full_size(size));
1660-
obj_cgroup_put(objcg);
16611659
}
16621660
}
16631661

0 commit comments

Comments
 (0)