Release DataFusion 52.3.0 (minor/) Release (Mar 2026)

[hareshkh]

Is your feature request related to a problem or challenge?

• Part of [EPIC] DataFusion Releases #19783
• Follow on to Release DataFusion 52.2.0 (minor/) Release (Feb 2026) #20287

Steps:

• Update version and changelog
• Test with DataFusion Python
• Test with DataFusion Comet
• Test with delta.rs:
• Test vortex:
• Test with iceberg-rust:
• Test with LakeSail:
• Test with parquet viewer
• Test with datafusion-materialized-views
• Voting Thread
• Ensure all fixes here also make their way to branch-53 - Release DataFusion 53.0.0 (Feb 2026 / Mar 2026) #19692 (comment)
• Publish to crates.io

Potential Content:

Requested Bug Fixes. Issue / requester / Backport

• Running Clickbench query 18 when spilling fails with "failed to fill whole buffer" error #19425 by @cetra3 / [branch-52] Fix Arrow Spill Underrun (#20159) #20684
• Repartition drops data when spilling #20683 by @xanderbailey / [branch-52] Fix repartition from dropping data when spilling (#20672) #20777
• fix: SortMergeJoin don't wait for all input before emitting #20482 by @mbutrovich @rluvaton / [branch-52] Backport fix: SortMergeJoin don't wait for all input before emitting #20482 #20699
• INNER JOIN with dictionary keys fails when run on parquet with pushdown_filters = true. #20696 by @alamb / [branch-52] fix: HashJoin panic with String dictionary keys (don't flatten keys) (#20505) #20708
• [branch-52] FFI_TableOptions are using default values only #20705 by @timsaucer / [branch-52] FFI_TableOptions are using default values only #20705
• "expected Dictionary(UInt16, Utf8) but found Utf8" while reading parquet files. #20041 by @alamb / [branch-52] Fix constant value from stats (#20042) #20709
• perf: Sort memory reservation causes performance regression with custom MemoryPool implementations #20728 by @mbutrovich / [branch-52] perf: sort replace free()->try_grow() pattern with try_resize() to reduce memory pool interactions #20732

[mbutrovich]

Comet could really use this fix in DF 52. #20482 @rluvaton will this be a straightforward backport?

[mbutrovich]

Comet could really use this fix in DF 52. #20482 @rluvaton will this be a straightforward backport?

• Did this: [branch-52] Backport fix: SortMergeJoin don't wait for all input before emitting #20482 #20699

[alamb]

Thank you @hareshkh for filing this -- this is awesome.

I am happy to help do the PMC only part of this (aka make a release candidate / do the official vote / publish to crates.io)

Selfishly we also found a new bug in our testing too so it is in my own self interest too :)

[alamb]

I would also like to include the fix for this into 52.3.0 if possible:

• INNER JOIN with dictionary keys fails when run on parquet with pushdown_filters = true. #20696

Once it gets merged I will make a backport to branch-52 as well

[comphead]

with releases and patches ongoing, I'm also wondering how to ensure some patch was backported to 52 from main but also missing from 53 🤔

[hareshkh]

Thanks @mbutrovich @alamb - I have included your suggested issues/PRs in the issue description as well.

@comphead - Thankyou for flagging! For now I can do a manual audit and ensure fixes that are on this issue are also backported to/part of branch-53 before closing this issue. Open to other ideas as well to better coordinate this!

For now, I have added a task in the this ticket's description - Ensure all fixes here also make their way to branch-53

[comphead]

For now, I have added a task in the this ticket's description - Ensure all fixes here also make their way to branch-53

Thank you @hareshkh

[timsaucer]

I can help shepherd this along because I have a PR I am about to put up that should get into 52 (and 53)

#20705

[alamb]

I also found that my reproducer for #20696 hit another bug (since fixed on main)

• Fix constant value from stats #20042
  I would like to include that PR as well in the patch release (I will make a backport PR)