Skip to content

Commit 8a127b8

Browse files
committed
Add bulk off-heap scoring for uint8 quantized vectors using Panama Vector API
Adds 4-wide bulk dot product and square distance operations for uint8 quantized vectors using the Panama Vector API with MemorySegment-based data access. This reduces reduceLanes calls by 4x compared to single- vector scoring, which helps under icache pressure during HNSW graph traversal. Key implementation details: - Uint8DotProduct and Uint8SqrDistance inner classes in MemorySegmentBulkVectorOps with platform-specific widening: - 128-bit: bytes -> shorts -> ints (2 convertShape parts) - 256-bit: bytes -> ints directly - 512-bit: bytes -> shorts -> ints (single part) - UINT8_NEEDS_PART1 guard prevents convertShape(..., 1) call on 512-bit where all 16 shorts fit in 16 ints in a single part - Int4 bulk scoring is explicitly not implemented; int4 scorers fall back to the existing single-vector Panama/Native paths via !isUint8() guard in bulkScoreBody Tests cover: basic uint8 (int7), large dimensions (128), odd dimensions (97), SIMD boundaries (15/16/17), tail paths (0-3 nodes), updateable scorer (MemorySegment query), and int4 fallback verification. Benchmarks (AMD Ryzen 7 7800X3D, AVX-512): - Float32 bulk: ~1.5x speedup (reference) - Uint8 bulk (clean icache): ~5-7% regression - Uint8 bulk (polluted icache): ~2x speedup Int4 bulk was evaluated but benchmarked ~2.3x slower than single-vector and has been removed from this PR.
1 parent 341976f commit 8a127b8

5 files changed

Lines changed: 1344 additions & 10 deletions

File tree

lucene/CHANGES.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -385,6 +385,8 @@ Improvements
385385

386386
Optimizations
387387
---------------------
388+
* GITHUB#16203: Add bulk off-heap scoring for uint8 quantized vectors using Panama Vector API. (Prithvi S)
389+
388390
* GITHUB#16176: Restore WANDScorer for TOP_SCORES + minShouldMatch > 1. (Tianxiao Wei)
389391

390392
* GITHUB#16153: Use TernaryLongHeap in UpdateGraphsUtils for faster HNSW graph merging. (Prithvi S)

0 commit comments

Comments
 (0)