fix(fuse): isolate partition and cluster layouts

[KKould]

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

• Remove the implicit semantic that prepended PARTITION BY expressions to CLUSTER BY keys
• Persist partition statistics independently from cluster statistics
• Keep compaction and reclustering selection, windows, and tasks within a single partition
• Preserve input order in TransformPartitionBy and only split blocks at adjacent partition boundaries
• Conservatively skip partition-aware reclustering when legacy metadata has no partition statistics

Tests

• Unit Test
• Logic Test
• Benchmark Test
• No Test - Explain why

Type of change

• Bug Fix (non-breaking change which fixes an issue)
• New Feature (non-breaking change which adds functionality)
• Breaking Change (fix or feature that could cause existing functionality not to work as expected)
• Documentation Update
• Refactoring
• Performance Improvement
• Other (please describe):

---

This change is 

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d6ea3e0da0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[KKould]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d6ea3e0da0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[sundy-li]

I found two issues that need to be addressed:

1. [P1] The full workspace test suite does not compile. After adding BlockMeta.partition_stats, the BlockMeta struct literal at src/query/storages/common/cache/src/manager.rs:1294 was not updated with the new field. The linux / check, linux / test_unit, and mac_check jobs all fail with E0063. I can also reproduce it locally with cargo test -p databend-storages-common-cache --lib --no-run.

2. [P2] Clustering introspection can still reuse legacy physical cluster statistics for partitioned tables. Historical cluster_stats.min/max vectors contain (partition keys..., cluster keys...), while clustering_information and clustering_statistics now prepare expressions for only the logical cluster key and still pass the same cluster_key_id to get_min_max_stats. Since that function returns the cached vectors whenever the key ID matches without validating their dimensions, clustering information may be displayed or calculated using the partition prefix. Please recompute from col_stats when the cached statistics dimensions do not match the logical expression count, or otherwise explicitly detect the legacy layout.

The focused Fuse test suite passes (cargo test -p databend-common-storages-fuse --lib: 103 tests), but it does not cover these two issues.

[KKould]

2. [P2] Clustering introspection can still reuse legacy physical cluster statistics for partitioned tables. Historical cluster_stats.min/max vectors contain (partition keys..., cluster keys...), while clustering_information and clustering_statistics now prepare expressions for only the logical cluster key and still pass the same cluster_key_id to get_min_max_stats. Since that function returns the cached vectors whenever the key ID matches without validating their dimensions, clustering information may be displayed or calculated using the partition prefix. Please recompute from col_stats when the cached statistics dimensions do not match the logical expression count, or otherwise explicitly detect the legacy layout.

This feature has not been deployed in the actual production environment, so I do not believe compatibility is required.