Commit fdab794
committed
Add bulk off-heap scoring for uint8 quantized vectors using Panama Vector API
Adds 4-wide bulk dot product and square distance operations for uint8
quantized vectors using the Panama Vector API with MemorySegment-based
data access. This reduces reduceLanes calls by 4x compared to single-
vector scoring, which helps under icache pressure during HNSW graph
traversal.
Key implementation details:
- Uint8DotProduct and Uint8SqrDistance inner classes in
MemorySegmentBulkVectorOps with platform-specific widening:
- 128-bit: bytes -> shorts -> ints (2 convertShape parts)
- 256-bit: bytes -> ints directly
- 512-bit: bytes -> shorts -> ints (single part)
- UINT8_NEEDS_PART1 guard prevents convertShape(..., 1) call on 512-bit
where all 16 shorts fit in 16 ints in a single part
- Int4 bulk scoring is explicitly not implemented; int4 scorers fall
back to the existing single-vector Panama/Native paths via !isUint8()
guard in bulkScoreBody
Tests cover: basic uint8 (int7), large dimensions (128), odd dimensions
(97), SIMD boundaries (15/16/17), tail paths (0-3 nodes), updateable
scorer (MemorySegment query), and int4 fallback verification.
Benchmarks (AMD Ryzen 7 7800X3D, AVX-512):
- Float32 bulk: ~1.5x speedup (reference)
- Uint8 bulk (clean icache): ~5-7% regression
- Uint8 bulk (polluted icache): ~2x speedup
Int4 bulk was evaluated but benchmarked ~2.3x slower than single-vector
and has been removed from this PR.1 parent 341976f commit fdab794
5 files changed
Lines changed: 1341 additions & 18 deletions
File tree
- lucene
- backward-codecs/src/test/org/apache/lucene/backward_codecs/lucene99
- benchmark-jmh/src/java/org/apache/lucene/benchmark/jmh
- core/src/java25/org/apache/lucene/internal/vectorization
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
385 | 385 | | |
386 | 386 | | |
387 | 387 | | |
| 388 | + | |
| 389 | + | |
388 | 390 | | |
389 | 391 | | |
390 | 392 | | |
| |||
0 commit comments