Skip to content

feat: GPU_HNSW / GPU_HNSW_SQ index using faiss::gpu::GpuIndexHNSW — native FP32/FP16/BF16/INT8 #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
devin-ai-integration wants to merge 24 commits into
base: main
Choose a base branch
from
Open

feat: GPU_HNSW / GPU_HNSW_SQ index using faiss::gpu::GpuIndexHNSW — native FP32/FP16/BF16/INT8 #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
8e24da1
feat: GPU_HNSW index using faiss::gpu::GpuIndexHNSW
devin-ai-integration[bot] Jul 9, 2026
b06fb32
fix: address Devin Review findings on GPU HNSW (vendored faiss)
premal Jul 11, 2026
452da1d
fix: address Devin Review findings on GPU_HNSW index node
premal Jul 11, 2026
a224f00
fix: declare gpu_ntotal_/gpu_dim_ mutable (written from const Search())
premal Jul 11, 2026
92e66b8
refactor: remove non-functional OCQ overflow queue from GPU HNSW
premal Jul 11, 2026
03a613f
feat: register GPU_HNSW_SQ as an alias of GPU_HNSW
premal Jul 11, 2026
18768e9
fix: mirror GPU HNSW re-review fixes into vendored faiss
premal Jul 11, 2026
0165f9d
feat: register native FP16/BF16 for GPU_HNSW and GPU_HNSW_SQ
premal Jul 11, 2026
5a9413c
fix: address GPU_HNSW review findings (gpu_ready_ reload, self-recall…
premal Jul 11, 2026
f4d7df3
test: add native int8 GPU_HNSW deserialize+search test
premal Jul 11, 2026
71c4c78
harden GPU HNSW: fail Deserialize on GPU upload error; mirror faiss r…
premal Jul 12, 2026
08ced66
fix(gpu_hnsw): quantized-cosine norms, reload lifetime safety, peak l…
premal Jul 12, 2026
5d20153
fix(gpu_hnsw): guard gpu_index_ with mutex instead of atomic<shared_ptr>
premal Jul 12, 2026
22fd143
test(gpu_hnsw): raise GPU HNSW cosine recall floor 0.65 -> 0.80
premal Jul 12, 2026
073344a
test(gpu_hnsw): CUDA upload fault-injection during Deserialize
premal Jul 12, 2026
bac5b70
refactor(gpu_hnsw): drop redundant h_node_ids copy (vendored faiss sync)
premal Jul 12, 2026
6906c99
fix(gpu_hnsw): run GPU upload on the DeserializeFromFile (mmap) load …
premal Jul 12, 2026
4d3d97d
fix(gpu_hnsw): size load estimate to fp32 resident graph; restore mma…
premal Jul 12, 2026
75474cd
revert(gpu_hnsw): drop MMAP eligibility; native int8 CPU copy is compact
premal Jul 12, 2026
7d8a448
fix: GPU_HNSW load estimate to compact int8 transient (~2x file), not…
premal Jul 12, 2026
db139ad
fix(gpu_hnsw): update load-resource test to match file_size*2 estimate
premal Jul 13, 2026
2f5a648
feat(gpu_hnsw): native int8 search via DP4A, bypass MockWrapper upcast
premal Jul 13, 2026
b5b1726
feat(gpu_hnsw): add DP4A dispatch in gpu_hnsw_search for INT8 dataset…
premal Jul 13, 2026
5a5c502
fix(gpu_hnsw): fix DP4A encoding bug and L2 guard
premal Jul 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions cmake/libs/libdiskann.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
add_definitions(-DKNOWHERE_WITH_DISKANN)
find_package(Boost REQUIRED COMPONENTS program_options)
include_directories(${Boost_INCLUDE_DIR})
find_package(Boost REQUIRED COMPONENTS program_options CONFIG)
include_directories(${Boost_INCLUDE_DIRS})
find_package(aio REQUIRED)
include_directories(${AIO_INCLUDE})
find_package(fmt REQUIRED)
Expand Down
23 changes: 23 additions & 0 deletions cmake/libs/libfaiss.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -593,3 +593,26 @@ if(__PPC64)
knowhere_utils)
target_compile_definitions(faiss PRIVATE FINTEGER=int)
endif()

# GPU HNSW CUDA sources — compiled when WITH_CUVS is enabled
if(WITH_CUVS)
set(FAISS_GPU_HNSW_SRCS
thirdparty/faiss/faiss/gpu/GpuIndexHNSW.cu
thirdparty/faiss/faiss/gpu/GpuIndex.cu
thirdparty/faiss/faiss/gpu/GpuResources.cpp
thirdparty/faiss/faiss/gpu/StandardGpuResources.cpp
thirdparty/faiss/faiss/gpu/impl/GpuHnswTypes.cu
thirdparty/faiss/faiss/gpu/impl/IndexUtils.cu
thirdparty/faiss/faiss/gpu/utils/DeviceUtils.cu
thirdparty/faiss/faiss/gpu/utils/StackDeviceMemory.cpp
thirdparty/faiss/faiss/gpu/utils/Timer.cpp
)
add_library(faiss_gpu_hnsw OBJECT ${FAISS_GPU_HNSW_SRCS})
target_include_directories(faiss_gpu_hnsw PRIVATE
${CMAKE_CURRENT_SOURCE_DIR}/thirdparty/faiss
${Boost_INCLUDE_DIRS}
)
target_compile_definitions(faiss_gpu_hnsw PRIVATE FINTEGER=int)
target_link_libraries(faiss_gpu_hnsw PRIVATE CUDA::cudart)
target_link_libraries(faiss PUBLIC faiss_gpu_hnsw)
endif()
2 changes: 2 additions & 0 deletions include/knowhere/comp/index_param.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ constexpr const char* INDEX_GPU_BRUTEFORCE = "GPU_BRUTE_FORCE";
constexpr const char* INDEX_GPU_IVFFLAT = "GPU_IVF_FLAT";
constexpr const char* INDEX_GPU_IVFPQ = "GPU_IVF_PQ";
constexpr const char* INDEX_GPU_CAGRA = "GPU_CAGRA";
constexpr const char* INDEX_GPU_HNSW = "GPU_HNSW";
constexpr const char* INDEX_GPU_HNSW_SQ = "GPU_HNSW_SQ";

constexpr const char* INDEX_HNSW = "HNSW";
constexpr const char* INDEX_HNSW_SQ = "HNSW_SQ";
Expand Down
10 changes: 9 additions & 1 deletion include/knowhere/index/index_static.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,16 @@
namespace knowhere {

struct Resource {
uint64_t memoryCost; // in bytes
uint64_t memoryCost; // retained host memory after load completes, in bytes
uint64_t diskCost; // in bytes
// Peak transient host memory required *during* load, in bytes. Defaults to 0
// so existing indexes are unaffected: consumers should fall back to their
// memoryCost-based heuristic when this is 0. Indexes whose peak load
// footprint differs from the retained footprint (e.g. GPU_HNSW frees its CPU
// copy after uploading to VRAM, so memoryCost≈0 but the peak covers the
// download buffer + deserialized CPU index + decode/graph staging) set this
// to the peak so the loader reserves enough host RAM and does not OOM.
uint64_t maxMemoryCost = 0;
};

#define DEFINE_HAS_STATIC_FUNC(func_name) \
Expand Down
8 changes: 8 additions & 0 deletions include/knowhere/index/index_table.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,14 @@ static std::set<std::pair<std::string, VecType>> legal_knowhere_index = {
{IndexEnum::INDEX_GPU_CAGRA, VecType::VECTOR_FLOAT16},
{IndexEnum::INDEX_GPU_CAGRA, VecType::VECTOR_INT8},
{IndexEnum::INDEX_GPU_CAGRA, VecType::VECTOR_BINARY},
{IndexEnum::INDEX_GPU_HNSW, VecType::VECTOR_FLOAT},
{IndexEnum::INDEX_GPU_HNSW, VecType::VECTOR_FLOAT16},
{IndexEnum::INDEX_GPU_HNSW, VecType::VECTOR_BFLOAT16},
{IndexEnum::INDEX_GPU_HNSW, VecType::VECTOR_INT8},
Comment thread
devin-ai-integration[bot] marked this conversation as resolved.
{IndexEnum::INDEX_GPU_HNSW_SQ, VecType::VECTOR_FLOAT},
{IndexEnum::INDEX_GPU_HNSW_SQ, VecType::VECTOR_FLOAT16},
{IndexEnum::INDEX_GPU_HNSW_SQ, VecType::VECTOR_BFLOAT16},
{IndexEnum::INDEX_GPU_HNSW_SQ, VecType::VECTOR_INT8},

// hnsw
{IndexEnum::INDEX_HNSW, VecType::VECTOR_FLOAT},
Expand Down
Loading