feat(partial-update): support ignore-delete semantics

[shyjsarah]

Purpose

Linked issue: close #572

Paimon Rust currently rejects ignore-delete options for merge-engine=partial-update tables and unconditionally fails on DELETE / UPDATE_BEFORE rows. This prevents compatibility with Java Paimon tables and historical files that use the supported ignore-delete semantics.

This change aligns the basic Rust partial-update implementation with Java for this option while keeping advanced partial-update modes unsupported.

Brief change log

• Accept exact ignore-delete and partial-update.ignore-delete options for partial-update tables, including explicit false.
• Ignore DELETE and UPDATE_BEFORE during read-side partial-update merging when enabled, including retract-only keys.
• Filter direct _VALUE_KIND retract rows before writer buffering, sequence allocation, data-file merge, and input changelog generation.
• Reject changing the effective option from true back to false, including alias removal and canonical-key override cases.
• Keep sequence groups, partial aggregation, remove-record options, and deletion-vector combinations fail-fast.

Tests

• cargo fmt --all -- --check
• cargo test -p paimon --lib
• cargo test -p paimon --test incremental_batch_scan_test
• cargo test -p paimon-datafusion --test pk_tables
• cargo clippy -p paimon --lib --tests -- -D warnings
• cargo clippy -p paimon-datafusion --test pk_tables -- -D warnings

Coverage includes:

• canonical and alias option parsing;
• explicit false;
• DELETE and UPDATE_BEFORE;
• retract-only keys;
• writer buffer, sequence, data, and input changelog behavior;
• ALTER attempts from effective true to false;
• core table and DataFusion SQL end-to-end reads.

API and Format

No public API or storage-format change. The change only enables an existing Java-compatible table option and filters retract rows according to that option.

Documentation

Updated the SQL table-option documentation with the supported keys, behavior, ALTER restriction, and remaining partial-update limitations.

[jerry-024]

[major] Preserve the one-way ignore-delete invariant across merge-engine changes

This guard only runs when the resulting schema is still partial-update, so the invariant can be bypassed across schema versions:

1. start with merge-engine=partial-update and ignore-delete=true
2. set merge-engine=deduplicate (this guard is disabled)
3. set ignore-delete=false or remove it (this guard is still disabled)
4. set merge-engine=partial-update again (the previous effective value is already false)

All calls to apply_changes succeed, and historical DELETE / UPDATE_BEFORE rows can become unreadable again. The Java SchemaManager.checkAlterTableOption enforces the true -> false restriction on the option itself, independent of merge engine.

Please preserve the effective true state across engine changes, or reject disabling/removing it whenever the current schema has it enabled. A regression test covering the four-step transition above would catch this.

[JingsongLi]

+1