Skip to content

Commit f8b56d0

Browse files
authored
PERF: Enlarge zlib read buffer in NIfTI reader for NAS performance (#1634)
The NIfTI reader opened files with a bare gzopen(), leaving zlib at its 8 KB default internal buffer. zlib refills from the underlying file in buffer-sized chunks, so a multi-GB .nii.gz triggered hundreds of thousands of small reads. On network attached storage each refill is a latency-bound round-trip, making reads disproportionately slow compared to a local disk.
1 parent ec22ac3 commit f8b56d0

3 files changed

Lines changed: 28 additions & 3 deletions

File tree

src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ReadNIfTIFile.cpp

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -378,6 +378,12 @@ Result<> ReadNIfTIFile::operator()()
378378
return MakeErrorResult(-34720, fmt::format("Could not open NIfTI file for reading: '{}'", m_InputValues->InputFilePath.string()));
379379
}
380380

381+
// Enlarge zlib's internal buffer before the first read/seek so the file is
382+
// refilled in large chunks instead of zlib's 8 KB default. This drastically
383+
// reduces the number of latency-bound network round-trips when the input
384+
// lives on a NAS. See nifti::k_GzReadBufferSize for the rationale.
385+
gzbuffer(gz, static_cast<unsigned int>(nx::core::nifti::k_GzReadBufferSize));
386+
381387
const z_off_t targetOffset = static_cast<z_off_t>(md.voxOffset);
382388
if(gzseek(gz, targetOffset, SEEK_SET) != targetOffset)
383389
{

src/Plugins/SimplnxCore/src/SimplnxCore/utils/NiftiUtilities.cpp

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -119,8 +119,10 @@ Result<NiftiMetadata> ReadNiftiHeader(const std::filesystem::path& filePath, boo
119119
return MakeErrorResult<NiftiMetadata>(-34700, fmt::format("Could not open NIfTI file for reading: '{}'", pathStr));
120120
}
121121

122+
gzbuffer(gz, k_GzReadBufferSize);
123+
122124
nifti_1_header hdr{};
123-
int bytesRead = gzread(gz, &hdr, static_cast<unsigned int>(k_HeaderSize));
125+
int bytesRead = gzread(gz, &hdr, k_HeaderSize);
124126
gzclose(gz);
125127

126128
if(bytesRead != static_cast<int>(k_HeaderSize))

src/Plugins/SimplnxCore/src/SimplnxCore/utils/NiftiUtilities.hpp

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@
2626
* `byteSwapRequired` from the metadata to do its own stream reads.
2727
*
2828
* The helpers here are deliberately independent of the simplnx filter
29-
* framework so they can be reused from unit tests, a future writer,
29+
* framework, so they can be reused from unit tests, a future writer,
3030
* or a command-line tool.
3131
*/
3232

@@ -36,7 +36,7 @@ namespace nx::core::nifti
3636
* @brief Size of the NIfTI-1 header in bytes. Always 348 for a valid
3737
* NIfTI-1 file regardless of byte order.
3838
*/
39-
inline constexpr usize k_HeaderSize = 348;
39+
inline constexpr unsigned int k_HeaderSize = 348;
4040

4141
/**
4242
* @brief Minimum legal value of the `vox_offset` field in a single-file
@@ -46,6 +46,23 @@ inline constexpr usize k_HeaderSize = 348;
4646
*/
4747
inline constexpr usize k_MinVoxOffset = 352;
4848

49+
/**
50+
* @brief Size (in bytes) of zlib's internal input/output buffer, set via
51+
* `gzbuffer()` immediately after every `gzopen()`.
52+
*
53+
* zlib defaults to an 8 KB internal buffer, which means it refills from
54+
* the underlying file in 8 KB chunks of compressed data — one `read()`
55+
* syscall each. On a local disk that is negligible, but on network
56+
* attached storage (NAS) every refill is a separate network round-trip,
57+
* so a multi-GB `.nii.gz` triggers hundreds of thousands of small,
58+
* latency-bound reads. Raising the buffer to 4 MiB cuts the number of
59+
* round-trips by ~512x while costing only a few × this size in working
60+
* memory (negligible next to the voxel volume). Larger values yield
61+
* diminishing returns once the buffer exceeds the link's
62+
* bandwidth-delay product and the client's own read-ahead.
63+
*/
64+
inline constexpr unsigned int k_GzReadBufferSize = 4194304;
65+
4966
/**
5067
* @brief Magic bytes that identify the single-file `.nii` / `.nii.gz`
5168
* format. Trailing null is included per the spec.

0 commit comments

Comments
 (0)