Commit 02a5d53
committed
libext: deep multi-window prefetch ring for ext4 sequential reads
The async double-buffered read-ahead only kept ~1-2 windows ahead of the
consumer, so a single sequential stream drove the NVMe queue to depth ~1-2
and topped out ~1.25 GB/s vs Linux's ~2.3 GB/s (Linux issues many concurrent
readahead requests, keeping the device queue deep).
Replace the two-window (cur/next) scheme with a ring of RA_WINDOWS windows
(256 KiB x 8 = 2 MiB per open file, same memory as before) filled by a pool
of RA_WORKERS worker pthreads. Each worker calls the existing synchronous
ext_internal_read() into a ring slot, so RA_WORKERS fills are in flight at
once -> device queue depth ~RA_WORKERS. As soon as the consumer drains a
window the ring slides forward and re-arms that slot for the window
RA_WINDOWS ahead, keeping the pipeline full.
Correctness: per-slot state (EMPTY/FILLING/READY) + a ring generation. All
worker-shared fields are under ra_cvmtx; the read path holds ra_lock (outer)
and takes ra_cvmtx to inspect slots. Workers never take ra_lock, so no
deadlock. Seek/write/truncate bump the generation and drain in-flight fills
before reusing buffers (no use-after-free); stale worker results whose
generation no longer matches are discarded. Reads larger than a window or
spanning two windows fall through to the direct read path. ~ext_vdata
broadcasts shutdown, joins all workers, then frees the buffers.
Reuses lwext4's existing bio path (ext_internal_read -> ext4_blocks_get_direct
-> one bio + bio_wait) rather than issuing raw bios, which would mean
re-implementing lwext4's extent->block mapping; a worker pool blocked in
bio_wait maps directly to device queue depth and keeps all that code correct.
Verified with tests/tst-ext4-bench.cc (absolute-offset byte verification) at
bs 4K/64K/128K/256K/262143/1M and files smaller than the ring: no VERIFY
FAIL.1 parent 2292f01 commit 02a5d53
1 file changed
Lines changed: 264 additions & 246 deletions
0 commit comments