Drop mimalloc (#6231)

When 6a29c2d ("mingw: use mimalloc",
2019-06-24) introduced the vendored mimalloc, the comparison was against
`nedmalloc` (which by then had not seen an upstream commit since 2014,
and whose repository was archived in 2019). The two were essentially at
parity in that benchmark; mimalloc was chosen because it was actively
developed. I do not really recall whether the platform's *default*
allocator was not part of the comparison; If it was, the performance was
still worse than mimalloc, if it wasn't, I forgot to test ;-)

Six years on, with `nedmalloc` safely on its way to being dropped from
the upstream codebase entirely
(gitgitgadget#2104, currently in `seen` as
e576abb), the question is no longer "mimalloc vs nedmalloc" but
"mimalloc vs the OS allocator". Re-running the same `git repack -adfq`
benchmark against each platform's current default allocator finds no
measurable speedup from mimalloc on any of Windows, macOS, or Linux.

## Methods

I recapitulated the same benchmark as cited in 6a29c2d (the original
comparison was nedmalloc vs mimalloc on `git repack -adfq` over a subset
of `linux.git`), now extended to the three GitHub-hosted runners
(`ubuntu-latest`, `macos-latest`, `windows-latest`). Each job built two
`git` binaries from the same source tree, vanilla and
`USE_MIMALLOC=YesPlease`, then prepared a fresh bare clone of
`linux.git` to a fixed `SHA`, and ran the repacks with both built `git`s
in randomized order for five iterations. Each iteration ran both
binaries exactly once on a freshly `copytree`-ed copy of the immutable
template repository; the order *within* an iteration was randomized so
any per-iteration confounder (cache state, runner warm-up, neighbour-VM
contention) would be shared symmetrically between variants. Timings
excluded the `copytree`. The full driver is the Python script
`ci/bench-mimalloc.py` on the [`mimalloc-benchmark`
branch](https://github.com/dscho/git/tree/mimalloc-benchmark/ci).

## Results: original `linux v2.6.20`-era workload (49,917 commits,
431,605 objects, ~204 MB pack)

| Platform | vanilla mean ± stdev | mimalloc mean ± stdev | Δ (mimalloc
− vanilla) |
|---|---|---|---|
| `ubuntu-latest` | 27.089s ± 0.060s | 27.041s ± 0.065s | −0.048s
(−0.18%) |
| `macos-latest` | 23.259s ± 1.206s | 25.076s ± 2.279s | +1.817s (+7.8%)
|
| `windows-latest` | 29.828s ± 1.651s | 30.329s ± 2.428s | +0.501s
(+1.7%) |

Workflow run: https://github.com/dscho/git/actions/runs/25374127848

## Results: 4x larger `linux v3.0` workload (255,039 commits, 2,059,429
objects, ~788 MB pack)

| Platform | vanilla mean ± stdev | mimalloc mean ± stdev | Δ (mimalloc
− vanilla) |
|---|---|---|---|
| `ubuntu-latest` | 134.723s ± **0.329s** | 134.801s ± **0.191s** |
+0.078s (+0.06%) |
| `macos-latest` | 130.183s ± 19.098s | 133.292s ± 18.991s | +3.109s
(+2.4%) |
| `windows-latest` | 145.183s ± 1.272s | 146.271s ± 4.161s | +1.088s
(+0.75%) |

Workflow run: https://github.com/dscho/git/actions/runs/25376885309

## Discussion

The Linux numbers on the larger workload are particularly clear: stdev
is below 0.3% of the mean for both variants, and the difference is well
inside that floor. Glibc's allocator and the vendored mimalloc are
statistically indistinguishable for `git repack -adfq` here.

`windows-latest` runners are noisier (per-run variance ~1-4%, mostly
neighbour-VM scheduling), but mimalloc never beats vanilla in either
workload. With the original justification for keeping a custom allocator
gone (the modern Windows segment-heap is no longer the slow
Windows-XP-era `HeapAlloc` that drove the original 2009 nedmalloc
adoption), there is nothing left to motivate the maintenance cost of a
vendored allocator.

`macos-latest` is too noisy at n=5 (stdev 14% of the mean) to draw a
firm conclusion, but the visible point-estimate goes the wrong way and
there is no plausible mechanism by which Apple's `libsystem_malloc`
would be slower than mimalloc.

## What this PR does *not* do

It does not by itself remove `nedmalloc` from the tree; that is still
promised as a follow-up of the in-flight upstream patch
gitgitgadget#2104, presently in `seen` as
e576abb. The first commit here is an `amend!` whose autosquashed
result is byte-identical to that upstream commit, so once the next
merging-rebase picks up the upstream patch the two will collapse
cleanly.

The five remaining `fixup!` reverts target each of the original
mimalloc-vendoring commits in reverse chronological order; once
autosquashed, the pairs cancel out to empty commits which the rebase
will drop, leaving the tree free of `compat/mimalloc/`, the
`USE_MIMALLOC` build infrastructure, and the supporting changes
(`compat/posix.h` `_DEFAULT_SOURCE` guard, `win32_pthread_*` renames)
that only existed to support the vendored allocator.

Author: dscho

Makefile

@@ -1524,7 +1524,6 @@ BUILTIN_OBJS += builtin/write-tree.o
 # upstream unnecessarily (making merging in future changes easier).
 THIRD_PARTY_SOURCES += compat/inet_ntop.c
 THIRD_PARTY_SOURCES += compat/inet_pton.c
-THIRD_PARTY_SOURCES += compat/mimalloc/%
 THIRD_PARTY_SOURCES += compat/nedmalloc/%
 THIRD_PARTY_SOURCES += compat/obstack.%
 THIRD_PARTY_SOURCES += compat/poll/%
@@ -2302,47 +2301,6 @@ ifdef USE_NED_ALLOCATOR
 	OVERRIDE_STRDUP = YesPlease
 endif
 
-ifdef USE_MIMALLOC
-	MIMALLOC_OBJS = \
-		compat/mimalloc/alloc-aligned.o \
-		compat/mimalloc/alloc.o \
-		compat/mimalloc/arena.o \
-		compat/mimalloc/bitmap.o \
-		compat/mimalloc/heap.o \
-		compat/mimalloc/init.o \
-		compat/mimalloc/libc.o \
-		compat/mimalloc/options.o \
-		compat/mimalloc/os.o \
-		compat/mimalloc/page.o \
-		compat/mimalloc/random.o \
-		compat/mimalloc/prim/prim.o \
-		compat/mimalloc/segment.o \
-		compat/mimalloc/segment-map.o \
-		compat/mimalloc/stats.o
-
-	COMPAT_CFLAGS += -Icompat/mimalloc -DMI_DEBUG=0 -DUSE_MIMALLOC --std=gnu11
-	COMPAT_OBJS += $(MIMALLOC_OBJS)
-
-$(MIMALLOC_OBJS): COMPAT_CFLAGS += -DBANNED_H
-
-$(MIMALLOC_OBJS): COMPAT_CFLAGS += \
-	-DMI_WIN_USE_FLS \
-	-Wno-attributes \
-	-Wno-unknown-pragmas \
-	-Wno-unused-function \
-	-Wno-array-bounds
-
-ifdef DEVELOPER
-$(MIMALLOC_OBJS): COMPAT_CFLAGS += \
-	-Wno-pedantic \
-	-Wno-declaration-after-statement \
-	-Wno-old-style-definition \
-	-Wno-missing-prototypes \
-	-Wno-implicit-function-declaration
-endif
-endif
-
-
 ifdef OVERRIDE_STRDUP
 	COMPAT_CFLAGS += -DOVERRIDE_STRDUP
 	COMPAT_OBJS += compat/strdup.o

compat/.gitattributes

@@ -1,2 +1 @@
 /zlib-uncompress2.c	whitespace=-indent-with-non-tab,-trailing-space
-/mimalloc/**/*	whitespace=-trailing-space