This repository contains the experimental codebase and evaluation scripts for our comprehensive study on disk-resident graph-based approximate nearest neighbor (ANN) search methods.
This repository includes implementations and evaluation scripts for the following disk-resident ANN systems:
-
AiSAQ: A scalable, DRAM-free approximate nearest neighbor search method with product quantization. Features inline PQ vectors, optimal vector rearrangement.
- Original repository: https://github.com/antifreeze53/aisaq-diskann
- See AiSAQ/README.md for detailed documentation
-
FreshDiskANN: Microsoft's DiskANN implementation with support for dynamic updates and streaming scenarios.
- Original repository: https://github.com/ShuiXianhua/FreshDISKANN
- See FreshDiskANN/README.md for detailed documentation
-
Gorgeous: High-performance disk-based vector search system with optimized data layout for large-scale high-dimensional datasets. This directory also includes implementations of Starling and DiskANN for comparative evaluation.
- Original repository: https://github.com/yinpeiqi/Gorgeous
- See Gorgeous/README.md for detailed documentation
-
PageANN: Page-level approximate nearest neighbor search system with disk-aligned graph organization to reduce random I/O.
- Original repository: https://github.com/Dingyi-Kang/PageANN
- See PageANN/README.md for detailed documentation
-
PipeANN: Low-latency, billion-scale, updatable graph-based vector store with ultra-low latency (<1ms) and efficient update mechanisms.
- Original repository: https://github.com/thustorage/PipeANN
- See PipeANN/README.md for detailed documentation
Each system has its own dependencies. Please refer to the individual README files in each subdirectory for specific requirements. Common dependencies include:
- CMake (v3.15+)
- C++ compiler (g++ or clang)
- Intel MKL
- Boost libraries
- libaio-dev (Linux)
- liburing-dev (for AiSAQ)
Please refer to the README file in each system's directory for specific build instructions:
- AiSAQ Build Instructions
- FreshDiskANN Build Instructions
- Gorgeous Build Instructions
- PageANN Build Instructions
- PipeANN Build Instructions
Detailed instructions for reproducing the experiments can be found in each system's documentation. Each system provides scripts and configuration files for running benchmarks and evaluations.
Each system in this repository retains its original license:
- AiSAQ: MIT License (Copyright Microsoft Corporation)
- FreshDiskANN: MIT License (Copyright Cong Fu, Changxu Wang, Deng Cai)
- Gorgeous: MIT License (Copyright Microsoft Corporation)
- PageANN: MIT License (Copyright Microsoft Corporation & Dingyi Kang)
- PipeANN: MIT License (Copyright Hao Guo)
Please refer to the LICENSE file in each subdirectory for full license text.