feat: GPU_HNSW / GPU_HNSW_SQ index using faiss::gpu::GpuIndexHNSW — native FP32/FP16/BF16/INT8

[devin-ai-integration]

Summary

GPU HNSW index implementation using a vendored FAISS GpuIndexHNSW backend. Accepts CPU-built HNSW/HNSW_SQ indexes at load time, uploads vectors and graph to GPU VRAM, frees the CPU copy, and runs a parallel beam search entirely on GPU with per-segment CUDA streams. Recall is tuned via ef.

Registered index types & precisions: GPU_HNSW and GPU_HNSW_SQ, each for fp32, fp16, bf16, and int8. GPU_HNSW_SQ is a thin alias — GpuHnswSQIndexNode subclasses GpuHnswIndexNode and only overrides Type(); the GPU search and load paths are identical.

Registrations (both index types) now cover all four precisions:

// index_table.h — legal (index, VecType) pairs
{GPU_HNSW, VECTOR_FLOAT}, {GPU_HNSW, VECTOR_FLOAT16},
{GPU_HNSW, VECTOR_BFLOAT16}, {GPU_HNSW, VECTOR_INT8},   // + same 4 for GPU_HNSW_SQ

// faiss_hnsw.cc
KNOWHERE_REGISTER_GLOBAL(GPU_HNSW, ...<fp16>..., fp16, true, feature::FP16 | feature::GPU);
KNOWHERE_REGISTER_GLOBAL(GPU_HNSW, ...<bf16>..., bf16, true, feature::BF16 | feature::GPU);
IndexStaticFaced<fp16>::Instance().RegisterStaticFunc<GpuHnswIndexNode>(INDEX_GPU_HNSW);
IndexStaticFaced<bf16>::Instance().RegisterStaticFunc<GpuHnswIndexNode>(INDEX_GPU_HNSW);
// (+ mirror for GPU_HNSW_SQ)

Native low-precision storage: fp16/bf16/int8 stay in their on-disk layout on the device (2/2/1 bytes/element) and are up-converted to fp32 per element inside the search kernel — mirroring the pre-existing INT8 path. The vendored FAISS copy (thirdparty/faiss/faiss/gpu/impl/GpuHnsw*.{h,cuh}) is kept byte-identical to 6si/faiss#1: a GpuHnswDatasetType enum drives kernel dispatch and upload_halfwidth_dataset() copies the raw 2-byte codes verbatim. The IndexNodeDataMockWrapper<fp16|bf16> converts fp16/bf16 query inputs to the node's fp32-facing interface, while the stored dataset stays native low-precision.

Load-path fix (mmap/file) — fixes the production querynode OOM: the eager GPU upload must run on both deserialization entrypoints. Milvus loads sealed segments through the mmap/file path (VectorMemIndex::LoadFromFile → index_.DeserializeFromFile()), not Deserialize(BinarySet). GpuHnswIndexNode previously only overrode Deserialize(BinarySet), so DeserializeFromFile fell through to the base CPU loader and the upload never ran: the CPU-built HNSW stayed resident in host RAM (fp32-expanded for int8/fp16/bf16 inputs, ~5.9 GiB/segment for mpd_v2) while VRAM sat idle, OOMKilling querynodes during the load storm before any search could trigger the lazy-upload fallback. Fix factors the upload/unpublish into helpers and calls them from both entrypoints:

Status DeserializeFromFile(filename, config) override {
    std::unique_lock lock(gpu_mutex_);
    UnpublishGpuIndexLocked();
    auto st = BaseFaissRegularIndexHNSWNode::DeserializeFromFile(filename, config);  // load CPU index
    if (st != Status::success) return st;
    return UploadCpuIndexToGpuLocked();  // copyFromWithMetric -> free indexes[0] -> publish gpu_index_
}

GpuHnswSQIndexNode inherits the fix. New [gpu_hnsw_p1] test loads via DeserializeFromFile (enable_mmap true/false) and asserts Size()==0 before any search (i.e. indexes[0] freed) — fails pre-fix and cannot be masked by the first-search lazy upload.

Architecture (unchanged core):

• GpuHnswIndexNode wraps faiss::gpu::GpuIndexHNSW; accepts CPU-serialized HNSW/HNSW_SQ at Deserialize() (binset key aliased from GPU_HNSW_SQ/HNSW_SQ/HNSW) and at DeserializeFromFile() (raw faiss index bytes / mmap). CPU copy freed after upload; Count()/Dim() captured before the free.
• Shared StandardGpuResources singleton (GetSharedGpuResources()); StaticEstimateLoadResource returns memoryCost=0; GetGpuConstructionMutex() serializes construction; gpu_ready_ publishes the built index for the lock-free search fast path; per-segment cudaStreamNonBlocking streams.

Tests: added Test GPU HNSW FP16 Deserialization and Test GPU HNSW BF16 Deserialization (CPU HNSW_SQ QT_fp16/QT_bf16 build → GPU load → recall vs brute force; floors 0.9 / 0.85), plus the [gpu_hnsw_p1] regression sections (cosine-norm parity, reload lifetime, load accounting, and the new DeserializeFromFile upload/free assertion). Guarded by KNOWHERE_WITH_CUVS, so they compile/run only on GPU builds (no silent no-op pass on CPU builds).

Kernel: 1-thread-per-distance, 128 concurrent candidates/block, num_parents == 0 stagnation break. The earlier non-functional "OCQ" overflow queue was removed here and in faiss#1 (decision record docs/design-docs/design_docs/20260711-gpu-hnsw-ocq-removal.md in 6si/milvus#6); use a larger ef for higher recall.

Filtered search: Search() rejects a non-empty bitset with invalid_args (delete/TTL/partition). No in-node CPU fallback (copy freed after upload); GPU-side bitset filtering is a documented follow-up. Scoped to append-mostly / immutable collections for now.

Verification: compile-verified via the GPU build. The DeserializeFromFile fix is validated by the [gpu_hnsw_p1] unit test and a live single-segment + full mpd_v2 load re-run (VRAM growth + host-anon-returns-to-baseline + zero OOMKills) — in progress. The prior perf numbers (8× g7e.2xlarge, mpd_v2 569M, GPU 3.3–4.5× CPU vec/s, R@1 0.892) predate this change and covered fp32/int8; native fp16/bf16 recall/latency/VRAM validation is pending GPU capacity — not yet validated.

Related PRs: 6si/faiss#1 (FAISS kernel), 6si/milvus#6 (Milvus integration).

Link to Devin session: https://6sense.devinenterprise.com/sessions/d39aba56b8a3467cbbf231ab631a06ed

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[ox-security]

Successfully scanned changes introduced in a pull request into main from gpu-hnsw-faiss.

Internal scan identifier: 04b82793-19d9-4c45-a031-92521fda40d1.

Total issues	Blocking issues	Scan status
1	0	✔️

Category	Issues
Code Security	1

See all issues found during this scan in the OX Security Application.

Detailed information

Issue #	1
Name	Unsafe Func • C - Testing Code • Taint Flow • Public Repo
Status	New
Enforcement	Monitor
Severity	Low
Category	Code Security
Source tools	OX Code Security
Recommendation	Validate the size of parameters at the function entry and use bounded counterparts (strncpy, strlcpy, snprintf, memcpy_s) instead of the unsafe APIs.

3 aggregations

File	Match
thirdparty/faiss/tests/test_read_index_deserialize.cpp	memcpy(writer.data.data() + offset, &corrupt_cs, sizeof(size_t));
thirdparty/faiss/tests/test_read_index_deserialize.cpp	memcpy(&buf[i + sizeof(uint8_t)], &bbs, sizeof(int));
thirdparty/faiss/tests/test_read_index_deserialize.cpp	memcpy(&buf[i + sizeof(int)], &qbs, sizeof(int));

[devin-ai-integration]

Devin Review found 2 new potential issues.

View 4 additional findings in Devin Review.

[devin-ai-integration]

Pushed 71c4c785 addressing an external code review of the GPU HNSW code:

• M1 (real bug) — GpuHnswIndexNode::Deserialize() returned Status::success even when the eager GPU upload threw, so a failed upload looked "loaded" and only surfaced on the first search. It now returns Status::cuda_runtime_error on upload failure and logs via LOG_KNOWHERE_ERROR_ (was fprintf(stderr,…)) so it lands in Milvus's structured logs.
• Mirrored the byte-identical vendored-faiss hardening (H1 scratch-release sync, L1/L2/C1/H5 docs) to keep 6si/faiss and the vendored copy in lockstep.

Previously-resolved review items (reset gpu_ready_ under lock on reload, self-recall test bound, fp16/bf16 index-table registration, const_cast/dtor rationale) remain in place. GPU unit tests (fp32/int8/fp16/bf16 + cosine) are running now on an L40S; I'll post measured recall numbers.

[devin-ai-integration]

Codex P1 review fixes — 08ced662

All three P1 findings addressed here (faiss side in 6si/faiss#1 2f5150915, milvus consumer in 6si/milvus#6 5e13c42763).

#1 Quantized-cosine inverse-norm semantics. CPU cosine records inverse L2 norms from the original input vectors (HasInverseL2Norms::get_inverse_l2_norms()). The GPU upload previously recomputed norms from decoded/quantized codes (and normalized decoded fp32 vectors in place), diverging from CPU scores/traversal for lossy SQ8/int8/fp16/bf16. Now the CPU index's stored inverse norms are uploaded and applied in the kernel; only flat-fp32 cosine (decoded == original) keeps normalizing in place. Vendored GpuHnswBuild.cuh kept byte-identical with faiss.

#2 Reload/search use-after-free race. gpu_index_ is now std::atomic<std::shared_ptr<GpuIndexHNSW>>; Search() takes a shared-ownership snapshot for the whole search, so a concurrent Deserialize() reload that resets/rebuilds the member cannot free an in-use index. Atomic readiness only guarantees visibility, not lifetime.

#3 Phase-separated load accounting. Resource gains maxMemoryCost (transient peak during load; defaults to 0 → other indexes keep their existing heuristic). GpuHnswIndexNode reports memoryCost=0 (CPU copy freed after upload) and maxMemoryCost covering download buffer + deserialized CPU index + decode/graph staging.

New tests (tests/ut/test_gpu_search.cc, tag [gpu_hnsw_p1]):

• Native low-precision device paths (fp16/bf16/sq8) via the non-mock fp32 node.
• SQ8/fp16/bf16 cosine CPU-vs-GPU parity asserting per-query top-1 id (≥0.95) and per-result score (rel-err <0.02) on same-direction / different-magnitude vectors — the direct feat: GPU HNSW index with OCQ kernel and INT8 support #1 validator.
• Search-vs-concurrent-reload lifetime stress (feat: GPU_HNSW / GPU_HNSW_SQ index using faiss::gpu::GpuIndexHNSW — native FP32/FP16/BF16/INT8 #2), meant to run under sanitizer.

• 4 concurrent searches with varying nq/k/ef vs own ground truth.

• Load-resource estimate assertions (memoryCost=0, maxMemoryCost≥peak) (feat: expose cuVS CAGRA num_random_samplings search param #3).

GPU build + L40S run of these is queued (v67); will post per-section results.

Still open (deferred, needs a prod test hook): cudaMalloc/memcpy fault-injection during Deserialize (assert no half-published index + retry). Tracked as follow-up.

[devin-ai-integration]

v67 L40S results — P1 fixes validated at runtime

Image gpu-hnsw-v67 (sha256:21a0818…3d8bfa02), knowhere 5d201532 + faiss vendored 2f5150915 + milvus 5e13c42763. Milvus segcore C++ compiled against the new Resource::maxMemoryCost.

New [gpu_hnsw_p1] case — 62/62 assertions pass:

• Native low-precision L2 (real device kernels, non-mock fp32 node): FP16 0.999, BF16 0.991, SQ8 0.992. First runtime validation of the native fp16/bf16/int8 search kernels — previous suite routed these through the fp32 mock wrapper.
• Quantized cosine parity (feat: GPU HNSW index with OCQ kernel and INT8 support #1 validator): top1_ratio=1.0 (≥0.95) and close_ratio=1.0 (≥0.98) for all of SQ8 / FP16 / BF16, on same-direction / different-magnitude vectors. Confirms uploading the CPU get_inverse_l2_norms() matches CPU scores/ordering.
• Search-vs-concurrent-reload (feat: GPU_HNSW / GPU_HNSW_SQ index using faiss::gpu::GpuIndexHNSW — native FP32/FP16/BF16/INT8 #2): PASS (plain pass — no TSAN/compute-sanitizer on the box; see caveat below).
• >4 concurrent searches: PASS, no crash.
• Resource estimate (feat: expose cuVS CAGRA num_random_samplings search param #3): memoryCost=0, maxMemoryCost=3072000 ≥ file_size+decode_staging=2048000; CPU HNSW leaves maxMemoryCost=0.

Existing GPU HNSW sections — 53/53 still pass: L2 0.973, Cosine 0.983, TopK 0.970/0.949/0.916, Roundtrip 0.999, SQ8 0.969, INT8 0.980, FP16 0.979, BF16 0.980.

Caveats / follow-ups:

• The reload race test passed but was not run under a thread sanitizer (none installed on the build box). The mutex-guarded snapshot (5d201532) makes the lifetime safe by construction, but a sanitizer pass is still worth doing before merge.
• cudaMalloc/memcpy fault-injection during Deserialize (Codex's 5th gap) remains deferred — needs a test-only fault hook in the upload path.

[devin-ai-integration]

Codex re-review should-fix #2 — raise GPU HNSW cosine recall floor (22fd143b)

"Test GPU HNSW Search Cosine Metric" used a 0.65 recall floor while the L2/IP/TopK sections use ~0.85. L40S measured cosine recall ~0.983, so 0.65 was far too loose to catch a regression. Raised to 0.80 to align with the other metrics.

Should-fix #3 (GpuIndexHNSW.h docstring listing fp16/bf16) was already in place — both the faiss and byte-identical vendored copies read Supports float32, fp16, bf16, and int8 (QT_8bit_direct_signed) data. (verified, no change needed).

v68 build will confirm the cosine section still passes at the 0.80 floor.