fix(btree): preserve empty keys in index metadata & disable global bitmap index

[lxy-9602]

Purpose

Linked issue: apache/paimon#7811

1. Align BTree index metadata with Java SortedIndexFileMeta and distinguish empty serialized keys from null key boundaries.

Previously, both an empty string key and a null boundary were encoded with a zero key length. Deserialization therefore treated an empty first key as null, which could cause incorrect file pruning or a null dereference when evaluating predicates.

This change:

• appends the Java-compatible format version and null-key flags to BTree index metadata;
• preserves zero-length keys as valid non-null keys;
• keeps backward compatibility with legacy metadata, including all-null index files;
• makes BTree file selection handle nullable metadata boundaries consistently with Java;
• adds end-to-end coverage for writing null, an empty string, and a normal string and querying them with different predicates.

2. Java Paimon has switched bitmap global indexes to a dedicated file format, replacing the previously shared wrapped bitmap file-index format. To prevent Paimon C++ from reading or writing incompatible bitmap global index files, this change temporarily treats the bitmap global index type as unsupported. The legacy implementation remains in the codebase and is enabled only by an internal testing option until support for the new Java-compatible format is implemented.

Tests

Added or updated coverage for:

• BTree metadata round trips with empty and null keys;
• legacy metadata deserialization;
• BTree file selection with an empty first key;
• BTree writer metadata containing null, empty-string, and normal keys;
• SDK end-to-end queries for:
  • equal and not equal;
  • is null and is not null;
  • in and not in;
  • less than and less than or equal;
  • greater than and greater than or equal.

API and Format

No public API under include/ is changed.

The BTree index metadata format is extended from:
firstKeyLength + firstKey + lastKeyLength + lastKey + hasNulls
to:
firstKeyLength + firstKey + lastKeyLength + lastKey + hasNulls + formatVersion + nullKeyFlags

The new format matches Java SortedIndexFileMeta. The reader remains backward compatible with legacy metadata that does not contain the two appended bytes.

Bitmap global indexes are temporarily disabled by default. Requests for the bitmap global index type are handled as an unknown index type. No bitmap implementation code is removed.

Documentation

Generative AI tooling

Generated-by: OpenAI Codex (GPT-5)

[lszskye]

+1