Skip to content

Commit 4092750

Browse files
committed
Update on "[Executorch][LLM] Use caching allocator for runner"
We observed that on iOS it improves perf by 6% because SDPA op does temp allocations. No significant difference on android though. Differential Revision: [D86120038](https://our.internmc.facebook.com/intern/diff/D86120038/) [ghstack-poisoned]
2 parents 494bbd5 + 997b5e2 commit 4092750

2 files changed

Lines changed: 3 additions & 3 deletions

File tree

extension/memory_allocator/cpu_caching_malloc_allocator.cpp

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,8 @@ void* CPUCachingAllocator::allocate(size_t size, size_t alignment) {
1919
return nullptr;
2020
}
2121
alignment = std::max(alignment, kCachingAllocatorDefaultAlignment);
22-
size_t adjusted_size = executorch::extension::utils::get_aligned_size(size, alignment);
22+
size_t adjusted_size =
23+
executorch::extension::utils::get_aligned_size(size, alignment);
2324
if (adjusted_size == 0) {
2425
return nullptr;
2526
}

extension/memory_allocator/memory_allocator_utils.h

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,7 @@ inline size_t get_aligned_size(size_t size, size_t alignment) {
2727
// this is the only portable way to get aligned memory from the heap.
2828
const size_t extra = alignment - 1;
2929
if ET_UNLIKELY (extra >= SIZE_MAX - size) {
30-
ET_LOG(
31-
Error, "Malloc size overflow: size=%zu + extra=%zu", size, extra);
30+
ET_LOG(Error, "Malloc size overflow: size=%zu + extra=%zu", size, extra);
3231
return 0;
3332
}
3433
size += extra;

0 commit comments

Comments
 (0)