Add fp16 (float16) vector encoding support - no quantization support by Pulkitg64 · Pull Request #16383 · apache/lucene

Pulkitg64 · 2026-07-10T16:52:04Z

Description

Introduce a new FLOAT16 VectorEncoding that stores vectors at half precision (16 bits per dimension, represented as short[]). This introduces the raw fp16 encoding only; scalar quantization of fp16 vectors is intentionally excluded for simplifying the PR and reviewing process. Will raise another PR with quantization support.

Initital full PR with both new encoding and quantization support: #15549

Introduce a new FLOAT16 VectorEncoding that stores vectors at half precision (16 bits per dimension, represented as short[]). Adds Float16VectorValues, KnnFloat16VectorField, KnnFloat16VectorQuery, short[] similarity and VectorUtil operations, codec reader/writer plumbing across the KNN vectors stack, and function-query value sources, together with tests. This introduces the raw fp16 encoding only; scalar quantization of fp16 vectors is intentionally excluded and will be added separately. GITHUB#15549

msokolov

Looks good! I have a bunch of tiny comments; mostly just double checking a few things and some javadoc stuff

msokolov · 2026-07-10T19:07:39Z

+ * limitations under the License.
+ */
+
+package org.apache.lucene.codecs.lucene95;


it seems kind of funny to be adding to the lucene95 codec, but I guess that's where the other OffHeap*VectorValues continue to live?

Yes, that was the motivation of creating the new OffHeapFloat16VectorValues but I am not sure about the best practice here. Should I move this new class to latest codec 104?

msokolov · 2026-07-10T19:09:13Z

+      long vectorDataLength,
+      IndexInput vectorData)
+      throws IOException {
+    if (configuration.docsWithFieldOffset == -2 || vectorEncoding != VectorEncoding.FLOAT16) {


seems weird that we respond to requests for other vectorEncoding by returning an empty values rather than raising an error -- is this consistent with the other encodings?

Yes this is consistent with other encodings as I copied this from OffHeapFloatVectorValues only. References:

OffHeapFloatVectorValues

OffHeapByteVectorValues

msokolov · 2026-07-10T19:10:12Z

+  }
+
+  /**
+   * Dense vector values that are stored off-heap. This is the most common case when every doc has a


maybe "This is the case" -- strike "most common" because it's the only one?

msokolov · 2026-07-10T19:10:58Z

+  private static class SparseOffHeapVectorValues extends OffHeapFloat16VectorValues {
+    private final DirectMonotonicReader ordToDoc;
+    private final IndexedDISI disi;
+    // dataIn was used to init a new IndexedDIS for #randomAccess()


"IndexedDISI"

msokolov · 2026-07-10T19:12:49Z

          switch (info.getVectorEncoding()) {
            case BYTE -> Byte.BYTES;
            case FLOAT32 -> Float.BYTES;
+            case FLOAT16 -> Short.BYTES;


tiny nit, but it would make me happy to list these switch statements in order of size: (BYTE, FLOAT16, FLOAT32); same throughout

msokolov · 2026-07-10T19:43:03Z

-  FLOAT32(4);
+  FLOAT32(4),
+
+  /** Encodes vector using 16 bits of precision per sample in IEEE floating point format. */


maybe "IEEE half-precision floating-point format"?

msokolov · 2026-07-10T19:59:53Z

+        for (int j = 0; j < v.length; j++) {
+          v[j] = Float.floatToFloat16(contents[i][j]);
+        }
+        doc.add(new KnnFloat16VectorField(field, v, EUCLIDEAN));


why EUCLIDEAN? But the case for float below doesn't specify any function?

Actually the float case is also using EUCLIDEAN.

public KnnFloatVectorField(String name, float[] vector) { this(name, vector, VectorSimilarityFunction.EUCLIDEAN); }

Let me check what happens if I use random Encoding.

msokolov · 2026-07-10T20:04:30Z

+  }
+
+  @Override
+  public FunctionValues getValues(Map<Object, Object> context, LeafReaderContext readerContext)


do we really need these float16 values sources? I wonder if we could use Float valuesSources over float16 vector fields?

do we really need these float16 values sources?

This was needed just for benchmark support in lucene util for exact scoring path(example ref: link). This is not needed in production path. Should we remove this change to make PR more simpler?

I wonder if we could use Float valuesSources over float16 vector fields?

I think we can do that and drop the ConstKnnFloat16ValueSource completely. It's just there to be consistent with other classes ConstKnnFloatValueSource and ConstKnnByteValueSource. But I don't have any strong opinion on this. Should we drop this??

OK I guess we support vector-valued values sources, which seems a bit weird to me -- what would consume them? But we may as well follow the existing pattern

msokolov · 2026-07-10T20:05:49Z


+  @Override
+  public Float16VectorValues getFloat16VectorValues(String field) throws IOException {
+    throw new UnsupportedOperationException("Float16 vectors not supported");


I wonder if it could support it? Not for this PR, but maybe it already has such a thing?

msokolov · 2026-07-10T20:07:25Z


  @Before
  public void init() {
    vectorEncoding = randomVectorEncoding();


does randomVectorEncoding now sometimes produce FLOAT16?

For all the backward codec, it doesn't because we have overridden this function with something like this:

@Override protected VectorEncoding randomVectorEncoding() { return random().nextBoolean() ? VectorEncoding.BYTE : VectorEncoding.FLOAT32; }

For for latest codecs we are producing Float16 encoding sometimes:

protected VectorEncoding randomVectorEncoding() { return VectorEncoding.values()[random().nextInt(VectorEncoding.values().length)]; }

msokolov · 2026-07-13T16:15:32Z

                beamWidth,
                infoStream,
                tinySegmentsThreshold);
+        case FLOAT16 ->


OK I just realized -- we should not be adding a writer method to an existing codec, because this will create a scenario where we can write an index with an older codec version (104?) that cannot be read by other versions of Lucene that have the 104 readers without fp16 support. When we add a new file format, we need a new Codec version, don't we?

after discussing and thinking about this some more, I realized this was wrong: Lucene doesn't expect to provide any forwards-compatibility guarantee. Even though the Codec version does not change, older releases are not expected to be able to read indexes produced by newer ones.

danmuzi

Thanks for your contribution! :)
I have a few comments.

danmuzi · 2026-07-14T18:40:04Z

+  public KnnFloat16VectorQuery(
+      String field, short[] target, int k, Query filter, KnnSearchStrategy searchStrategy) {
+    super(field, k, filter, searchStrategy);
+    this.target = target;


The float32 implementation validates the target vector as follows:

this.target = VectorUtil.checkFinite(Objects.requireNonNull(target, "target"));

This class does not have any validation for target, so it can cause several issues.

If target is null, NPE is thrown later during search from DefaultFlatVectorScorer.getRandomVectorScorer() when accessing target.length.

If the target contains values such as 0x7C00(+Infinity), 0xFC00(-Infinity), or 0x7E00 (NaN), similarityFunction.compare() may return NaN.
In TopKnnCollector, comparisons involving NaN return false, so the search may silently return incorrect top-K results without throwing an exception.
This is only caught when assertions are enabled, by the assert Float.isFinite(result) check around VectorUtil.dotProduct().
In production, the issue may remain undetected in production level.

If target.length is 0, KnnFloat16VectorQuery.toString() throws an ArrayIndexOutOfBoundsException because it accesses target[0].

So I think adding VectorUtil.checkFiniteFloat16() would be a reasonable solution.

// VectorUtil.java public static short[] checkFiniteFloat16(short[] v) { for (int i = 0; i < v.length; i++) { if ((v[i] & 0x7C00) == 0x7C00) { throw new IllegalArgumentException( "non-finite float16 value at vector[" + i + "]=" + Float.float16ToFloat(v[i])); } } return v; } // KnnFloat16VectorQuery.java this.target = VectorUtil.checkFiniteFloat16(Objects.requireNonNull(target, "target"));

Fixed in the next revision

danmuzi · 2026-07-14T18:42:04Z

+      String name,
+      short[] vector,
+      VectorSimilarityFunction similarityFunction,
+      VectorEncoding vectorEncoding) {


It seems not used.

Yeah, I already removed it in my next revision. Thanks for noticing it.

danmuzi · 2026-07-14T18:57:47Z

+   * Creates a numeric vector field. Fields are single-valued: each document has either one value or
+   * no value. Vectors of a single field share the same dimension and similarity function. Note that
+   * some vector similarities (like {@link VectorSimilarityFunction#DOT_PRODUCT}) require values to
+   * be unit-length, which can be enforced using {@link VectorUtil#l2normalize(float[])}.


Following up on Michael's comment, VectorUtil does not have a l2normalize method for short[].

I have removed the comment for now, even float32 is not exercising this path (l2normalize) in without quantization path i.e based on the code it looks like its the user responsibility to l2normalize the vectors. Once we add the quantization support for float16, i will again add the support for l2 normalize

danmuzi · 2026-07-14T19:15:06Z

+  public KnnFloat16VectorField(
+      String name, short[] vector, VectorSimilarityFunction similarityFunction) {
+    super(name, createType(vector, similarityFunction));
+    fieldsData = vector; // null check done above


This should also validate the vector using the VectorUtil.checkFiniteFloat16() suggested in the comment on KnnFloat16VectorQuery.

fieldsData = VectorUtil.checkFiniteFloat16(vector);

Without this validation, the following issues may occur:

When the segment is flushed, the HNSW graph builder computes similarity scores with these values, and the results become NaN.
NaN breaks the neighbor selection logic, so the graph connections around those nodes end up more or less random.

Once these values are written into a segment, every later merge reuses them to rebuild the graph, so the damage spreads to new segments as well.

No error or warning is raised in production.

Thanks for pointing it out. I have fixed it in next revision.

danmuzi · 2026-07-14T19:15:49Z

+      VectorSimilarityFunction similarityFunction,
+      VectorEncoding vectorEncoding) {
+    super(name, createType(vector, similarityFunction));
+    fieldsData = vector; // null check done above


These functions were unused so I have removed them in the next revision.

danmuzi · 2026-07-14T19:16:00Z

+      throw new IllegalArgumentException(
+          "The number of vector dimensions does not match the field type");
+    }
+    fieldsData = vector;


danmuzi · 2026-07-14T19:16:08Z

+      throw new IllegalArgumentException(
+          "value length " + value.length + " must match field dimension " + type.vectorDimension());
+    }
+    fieldsData = value;


jeho-rpls

I was reading this PR out of interest in the encoding and merge paths and left two small inline comments. Thanks!

jeho-rpls · 2026-07-16T11:24:13Z

+        case FLOAT16:
+          Float16VectorValues val = context.reader().getFloat16VectorValues("field");
+          vectorScorer = val.scorer(new short[] {1, 2});
+          break;


If I am reading this right, these shorts are interpreted as fp16 bit patterns, so {1, 2} would decode to a nearly zero vector rather than {1.0, 2.0} like the byte and float cases.

I think the test still passes because it only counts the matched docs and never checks scores. Would it make sense to use like Float.floatToFloat16(1f) and Float.floatToFloat16(2f) here instead?

thanks for noticing, fixed it.

jeho-rpls · 2026-07-16T11:26:42Z

+  public RandomVectorScorer getRandomVectorScorer(
+      VectorSimilarityFunction similarityFunction, KnnVectorValues vectorValues, short[] target)
+      throws IOException {
+    return null;


Minor one, but returning null here could surface as an NPE at some caller far from the cause.

I noticed Lucene102BinaryQuantizedVectorsReader in this package throws UnsupportedOperationException for the same situation. Would it be more
consistent to throw here as well?

Fixed in next revision

danmuzi

Looks good to me :)

github-actions Bot added module:core/store module:core/index module:core/search module:highlighter module:core/codecs module:test-framework module:sandbox module:queries module:core/hnsw labels Jul 10, 2026

Pulkitg64 mentioned this pull request Jul 10, 2026

Add half-float (FP16) storage support for vectors #15549

Open

Pulkitg64 added 2 commits July 10, 2026 17:11

Fixed typo

cea9b63

Update the github change id in CHANGES.txt

2a6f9e1

msokolov reviewed Jul 10, 2026

View reviewed changes

msokolov reviewed Jul 13, 2026

View reviewed changes

danmuzi requested changes Jul 14, 2026

View reviewed changes

Addressed comments: fixed comments and java docs

2a839c5

jeho-rpls reviewed Jul 16, 2026

View reviewed changes

Addressed some more comments

497631d

Pulkitg64 requested review from danmuzi, jeho-rpls and msokolov July 17, 2026 04:27

danmuzi approved these changes Jul 18, 2026

View reviewed changes

Uh oh!

Conversation

Pulkitg64 commented Jul 10, 2026

Description

Uh oh!

msokolov left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danmuzi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeho-rpls left a comment

Choose a reason for hiding this comment

Uh oh!