Increase the throughput of the validate_duplicate_files#2296

tomighita wants to merge 4 commits intoapache:mainapache/iceberg-rust:mainfrom

dbt-labs:tomighita/increase-duplicate-check-throughputdbt-labs/iceberg-rust:tomighita/increase-duplicate-check-throughputCopy head branch name to clipboard