Warnings when compiled with GCC

[MIvanchev]

Thanks for this awesome piece of software that I hope to use A LOT in the future. Congratulations on the IMMACULATE selection of compile / link flags, this is what responsibility looks like! I saw these 2 warnings while compiling with GCC here, maybe you are interested :)

include/buddy_alloc/buddy_alloc.h: In function 'depth_for_size':
include/buddy_alloc/buddy_alloc.h:1121:13: warning: conversion to 'size_t' {aka 'unsigned int'} from 'int' may change the sign of the result [-Wsign-conversion]
 1121 |     depth = count_trailing_zeroes(effective_memory_size) + 1
      |             ^~~~~~~~~~~~~~~~~~~~~
include/buddy_alloc/buddy_alloc.h: In function 'buddy_tree_fragmentation':
include/buddy_alloc/buddy_alloc.h:1998:28: warning: conversion from 'size_t' {aka 'unsigned int'} to 'unsigned char' may change value [-Wconversion]
 1998 |     return fractional_mask - (quality_percent & fractional_mask);

gcc (GCC) 14.2.1 20250405
Copyright (C) 2024 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE

[spaskalev]

Hey, thanks for the kind words. I feel that those flags are the only sane way to use the compiler, anything else is just too risky for code that should work on and on for years without failure. That being said, fancy making a pull request ? The allocator has been working fine for a while now but I rather have more people familiar with it and not being a single point of failure.

[MIvanchev]

I feel that those flags are the only sane way to use the compiler, anything else is just too risky for code that should work on and on for years without failure.

Man, are you me?! I have felt for a long time that this is the correct way to compile anything C at all... In fact, I am preparing a (modern) C talk for the next ACCU and would cite buddy_alloc as a reference for proper compiler configuration :)

That being said, fancy making a pull request ?

Not at all.

The allocator has been working fine for a while now but I rather have more people familiar with it and not being a single point of failure.

Yes, I am testing it now for a fairly complex project and worked as substitute for glibc malloc immediately. I only wrapped it in a mutex for multithreading (project is heavily multithreaded):

#define do_with_mutex(op) \
    do { \
        pthread_mutex_lock(&g_mem.mutex); \
        op; \
        pthread_mutex_unlock(&g_mem.mutex); \
    } while(false)

__attribute__((malloc, malloc (mem_free, 1)))
__attribute__ ((alloc_size (1)))
void *mem_malloc(size_t size)
{
    void *ret;
    // ret = malloc(size);
    do_with_mutex(ret = buddy_malloc(g_mem.buddy, size));
    return ret;
}

So far absolutely no issues, I can barely believe it!

[spaskalev]

I fuzzed it for months and it has been running in production, on multiple deployments for several years now. The allocation functionality is solid and all the recent issues have been in the arena resizing code, which arguably is way more complex that the actual allocation.

[spaskalev]

As a replacement of general purpose malloc - be careful. Modern libc implementations will often do stuff like guard pages on the side of large allocations to trap on out of bounds reads and writes or do stuff like valgrind does like fill the memory with some bit pattern and then check it for changes. This library, being lightweight and streamlined does not have such capabilities.

[oreyg]

Yes, I am testing it now for a fairly complex project and worked as substitute for glibc malloc immediately. I only wrapped it in a mutex for multithreading (project is heavily multithreaded):

Hate to be that guy - glibc allocator is usually solid. Buddy alloc is a great fit for an arena allocator, or when you have no heap at all, say on embedded devices. As a general purpose allocator I doubt you would see immediate wins. Please keep us posted, I'm curious what results you would have.

[MIvanchev]

I am already seeing a huge win in that it's a single header gizmo I can compile with my own flags and link statically and does all I need and not an ounce more.

Modern libc implementations will often do stuff like guard pages on the side of large allocations to trap on out of bounds reads and writes or do stuff like valgrind does like fill the memory with some bit pattern and then check it for changes.

I am poisoning / unpoisoning with the sanitizer API manually. Of course I lose some of that flexibility, but I really have 0 choice in this, glibc's allocator simply doesn't cover the requirements here. It's just not configurable enough.

[oreyg]

I am already seeing a huge win in that it's a single header gizmo I can compile with my own flags and link statically and does all I need and not an ounce more.:

Again - hate to be that guy - if you'd start missing some features - try mimalloc. It has all the features of the modern general purpose allocator, but is still simple enough.

[spaskalev]

Just a bit of trivia - the original reason I started this project was that I needed an allocator that is serializable - like it can be copied around (even across machines), persisted to and restored from disk. Not a single one of the popular allocators had this so I ended up writing this one with that goal in mind.

[MIvanchev]

@oreyg what would be some features that I will be missing with buddy_alloc compared to mimalloc assuming the behavior is correct?

[spaskalev]

Proper multi-threading for starters. Sure, you can guard the buddy behind a lock but that can very quickly become a point of contention. Modern multi-threaded allocators keep per-thread arenas.

Then frequent small allocations. Due to the tree traversal mixing a big arena with very large number of small allocation is the worst way to use the buddy as you will be doing deep tree traversals. Of course you can work around this by nesting buddy arenas but why bother if a good allocator can do it behind the scenes.

This piece of code is spartan. Or I guess you could say unix-y. It does one thing and it does it well.

[spaskalev]

This is probably one of the few projects where maintainers are actively trying to dissuade a user from using it :))

[oreyg]

@oreyg what would be some features that I will be missing with buddy_alloc compared to mimalloc assuming the behavior is correct?

Buddy alloc is by design a simpler allocator.
In practice that means - for multithreaded cases you would need a lock - which defeats one the best buddy-alloc advantages - predictable alloc time. Mimalloc is lockless.

Then, the page guards and out-of-bound detection - mimalloc has it out of the box.

Then, mimalloc tries to keep allocations that are close in time close in memory, this has performance implications, if your program allocates and frees a lot, and runs for hours (say, you have a game).

[MIvanchev]

Let's run them both and see then :) So far I am seeing no difference at all but I am worrying about the noticeability of bugs beyond the sanitizer. So far that's my only concern and I prefer to start simple and build up incrementally but you do have a point hehe. Still happy I found something that's so easy to integrate..

[spaskalev]

If you want to use the buddy effectively in a multi-threaded context consider a meta allocator approach. That's what all modern allocators do. Have multiple arenas, some of them per thread and some of them global. Then threads can allocate locally without locks unless full at which case an arena can be swapped for one of the other ones.