Search before asking
Please describe the bug
On the latest main commit a9a8f6b, GlobalIndexScanner converts missing or unparseable sorted global-index metadata into BTreeIndexMeta(None, None, false). BTreeIndexMeta then interprets that value as an only-null index.
For equality and range predicates, file-level pruning reports no match without opening the physical index file. The scanner returns an indexed result such as Some([]) instead of None, so rows covered by that index shard can be silently omitted in fast, full, and detail search modes.
A minimal reproduction uses two BTree shards for the same field: one with valid metadata and one with missing metadata. Querying a key present in both shards returns only the valid shard result instead of falling back to the normal table scan.
Malformed key lengths are related: BTreeIndexMeta::deserialize currently slices with unchecked lengths and can panic before the scanner can handle the parse error.
Solution
Preserve missing or unparseable metadata as unavailable and return None when evaluating that field, allowing the caller to use the normal table scan. Validate serialized key lengths before slicing so malformed metadata returns InvalidData instead of panicking.
Are you willing to submit a PR?
Search before asking
Please describe the bug
On the latest main commit a9a8f6b, GlobalIndexScanner converts missing or unparseable sorted global-index metadata into BTreeIndexMeta(None, None, false). BTreeIndexMeta then interprets that value as an only-null index.
For equality and range predicates, file-level pruning reports no match without opening the physical index file. The scanner returns an indexed result such as Some([]) instead of None, so rows covered by that index shard can be silently omitted in fast, full, and detail search modes.
A minimal reproduction uses two BTree shards for the same field: one with valid metadata and one with missing metadata. Querying a key present in both shards returns only the valid shard result instead of falling back to the normal table scan.
Malformed key lengths are related: BTreeIndexMeta::deserialize currently slices with unchecked lengths and can panic before the scanner can handle the parse error.
Solution
Preserve missing or unparseable metadata as unavailable and return None when evaluating that field, allowing the caller to use the normal table scan. Validate serialized key lengths before slicing so malformed metadata returns InvalidData instead of panicking.
Are you willing to submit a PR?