Skip to content

Support materialized partial-update and aggregation reads with deletion vectors #579

Description

@shyjsarah

Motivation

Paimon Rust can plan deletion vectors and apply them in DataFileReader, but primary-key tables using partial-update or aggregation are always routed to KeyValueFileReader. That reader rejects splits carrying deletion vectors, so otherwise readable compacted plans fail with KeyValueFileReader does not support deletion vectors.

The requested scope is read-only compatibility for files whose merge result is already fully materialized.

Solution

Route the whole plan through DataFileReader only when deletion vectors are enabled, deletion-vectors.merge-on-read is disabled, every split is raw-convertible, every file is above level 0, and every file has a known zero delete_row_count.

Keep failing closed for level-0 files, ambiguous metadata, mixed plans that still need per-key merging, merge-on-read, and deletion-vector writes.

Anything else?

This is independent of #572 and #573.

Willingness to contribute

  • I am willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions