feat(diskann): add macOS POSIX AIO with kqueue completion#605
Open
iaojnh wants to merge 83 commits into
Open
Conversation
… into refactor/diskann_dlopen
… into refactor/diskann_dlopen
# Conflicts: # .github/workflows/03-macos-linux-build.yml # .github/workflows/nightly_coverage.yml # CMakeLists.txt # examples/c++/CMakeLists.txt # python/tests/detail/fixture_helper.py # python/tests/test_collection_diskann.py # python/zvec/__init__.py # src/binding/python/model/common/python_config.cc # src/core/CMakeLists.txt # src/core/algorithm/diskann/CMakeLists.txt # src/core/algorithm/diskann/diskann_builder.cc # src/core/algorithm/diskann/diskann_file_reader.cc # src/core/algorithm/diskann/diskann_file_reader.h # src/core/algorithm/diskann/diskann_searcher.cc # src/core/algorithm/diskann/diskann_streamer.cc # src/db/index/common/schema.cc
iaojnh
changed the title
iaojnh
marked this pull request as ready for review
July 20, 2026 13:18
iaojnh
requested review from
Cuiyus,
chinaux,
egolearner,
feihongxu0824,
richyreachy and
zhourrr
as code owners
July 20, 2026 13:18
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relationship to #557
This PR supersedes #557 and contains its head commit. It preserves the Linux support introduced there, while updating the branch to the current
mainand replacing the macOS I/O prototype with a complete POSIX AIO implementation.The main difference is the role of kqueue:
EVFILT_READ, but still executed every read synchronously usingpread(). Therefore, kqueue acted only as a readiness wrapper and did not provide asynchronous disk I/O.aio_read()and usesSIGEV_KEVENT/EVFILT_AIOthroughkevent64()for completion notification.The new implementation batches requests, validates every completion, safely drains outstanding AIO operations on failure, and falls back to synchronous
pread()when POSIX AIO or kqueue cannot be used.It also limits this path explicitly to macOS, exposes
POSIX_AIOas a distinct backend through the C and Python APIs, and adds tests that verify both the actual POSIX AIO path and the synchronous fallback.