Memory seemingly isn't reclaimed unless `MIMALLOC_PURGE_DELAY=0` is set, on linux

[jberryman]

It seems as though memory is not being freed back to the OS (sometimes? always?).

I'm not an expert and may be missing some subtleties, but the latest docs suggest MADV_DONTNEED is used by default and that unused pages are "purged" after a default brief delay. But it seems as though memory might never be reclaimed (from the point of view of RES in htop), unless MIMALLOC_PURGE_DELAY=0 is set.

Some related context might be this old issue, although I can't anymore say whether that issue wasn't in fact this issue.

EDIT: this was with mimalloc 0.1.46

Repro

I recommend running htop in one terminal, and this in another (note the process name, which you might need to change):

$ watch 'cat /proc/$(pidof mimalloc_test)/smaps | grep -i "anonymous\|referenced\|active\|lazyfree\|kernelpages" | sort -k2,2nr | head -n 5'

You can then try various flags:

$ ./target/release/mimalloc_test small                                  # borked
$ MIMALLOC_PURGE_DELAY=0 ./target/release/mimalloc_test small           # fine
$ MIMALLOC_PURGE_DELAY=0 ./target/release/mimalloc_test                 # fine
$ MIMALLOC_PURGE_DELAY=1 ./target/release/mimalloc_test small           # borked
$ MIMALLOC_PURGE_DELAY=0 MIMALLOC_PURGE_DECOMMITS=0 ./target/release/mimalloc_test  # fine, in the sense that we see memory
                                                                                    # move to the `LazyFree` column promptly

I did not observe any difference with small or without

Cargo.toml

[package]
name = "mimalloc_test"
version = "0.1.0"
edition = "2021"

[dependencies]
mimalloc = { version = "0.1" }

main.rs

use mimalloc::MiMalloc;

#[global_allocator]
static GLOBAL: MiMalloc = MiMalloc;

use std::{env, thread, time::Duration};

fn main() {
    let args: Vec<String> = env::args().collect();
    let use_small = args.get(1).map(|s| s == "small").unwrap_or(false);

    let mut blocks = Vec::new();

    if use_small {
        println!("Allocating ~1 GB using 8 KB chunks...");
        for _ in 0..131072 {
            let block = vec![0u8; 8 * 1024];
            blocks.push(block);
        }
    } else {
        println!("Allocating ~1 GB using 8 MB chunks...");
        for _ in 0..128 {
            let mut block = vec![0u8; 8 * 1024 * 1024];
            for i in (0..block.len()).step_by(4096) {
                block[i] = 1; // touch each page so it shows up in RES for default allocator
            }
            blocks.push(block);
        }
    }

    println!("Sleeping 5s: observe RES in `top`...");
    thread::sleep(Duration::from_secs(5));

    println!("Freeing memory...");
    drop(blocks);

    println!("Sleeping 30s: watch RES drop in `top` if memory is returned to OS...");
    thread::sleep(Duration::from_secs(30));
}

[octavonce]

Can you please check with the latest version? The latest release from upstream says it fixed bugs about memory not being freed.

[jberryman]

I don't observe any difference in behavior between 0.1.46 and 0.1.47

[Borderliner]

@jberryman Does it also affect v3?

[jberryman]

Please try the reproducer; I'm not actively working on this right now.