Skip to content

fix(query): support delete by internal columns#19977

Draft
dantengsky wants to merge 2 commits into
databendlabs:mainfrom
dantengsky:fix-delete-internal-columns-meta-only
Draft

fix(query): support delete by internal columns#19977
dantengsky wants to merge 2 commits into
databendlabs:mainfrom
dantengsky:fix-delete-internal-columns-meta-only

Conversation

@dantengsky

@dantengsky dantengsky commented Jun 8, 2026

Copy link
Copy Markdown
Member

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Fix DELETE predicates on Fuse internal metadata columns so _segment_name and _block_name can be used for metadata-only delete pruning instead of reaching row-level mutation execution and panicking.

This supports statements like:

DELETE FROM t WHERE _block_name IN (...);
DELETE FROM t WHERE _segment_name IN (...);
DELETE FROM  t WHERE _segment_name = 's1' and _segment_name = 's2';

Known limitation: meta-only DELETE currently supports _block_name / _segment_name with constant IN lists only; IN (SELECT ...) subqueries are not supported yet and may fail through the existing mutation subquery path.

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

Support metadata-only DELETE pruning for _segment_name and _block_name predicates while keeping internal column predicates out of row-level mutation evaluation.

Tests: cargo fmt --all; cargo check -p databend-query; cargo build --bin databend-query --bin databend-meta --bin databend-sqllogictests; target/debug/databend-sqllogictests --handlers http --run_file 09_0053_delete_by_internal_columns.test
@github-actions github-actions Bot added the pr-bugfix this PR patches a bug in codebase label Jun 8, 2026
@dantengsky

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d4a78d215e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +376 to +377
if !has_column_ref || !internal_only || !expr.is_deterministic(&BUILTIN_FUNCTIONS) {
row_predicates.push(predicate.clone());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Strip internal columns before building row filters

When a DELETE predicate mixes an internal column with a real column (for example DELETE ... WHERE _block_name = $b AND a = 1), this branch copies the original predicate into row_filters. Just below, read_partition_columns filters internal columns out of the read schema, and add_mutation_source projects the row filter with schema.index_of(name).unwrap(), so _block_name/_segment_name is no longer present and the query panics instead of deleting the matching rows. The block/segment pruning filter can still use the full predicate, but the per-row filter must not retain references to internal columns unless the mutation source also materializes them.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-bugfix this PR patches a bug in codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant