"I would incline for 53.1.0 if there is no stoppers" Last week's DataFusion call discussed the challenges the community is having shipping parallel releases (52.3.0 and 53.0.0 last week). We are already working on a 52.4.0 release https://github.com/apache/datafusion/issues/20855. I am not in favor of releasing 53.0.0 with the expectation of a 53.1.0 due to issues found in the RC phase.
I remain -1 (non-binding) on releasing this rc2 as 53.0.0. On Tue, Mar 17, 2026 at 11:36 AM Oleks V. <[email protected]> wrote: > Merged the fix for CI, I can create RC3 today, the question should we wait > for updated arrow in this release or wait for 53.1.0? I would incline for > 53.1.0 if there is no stoppers > > On 2026/03/17 14:48:41 Matt Butrovich wrote: > > Just to clarify: the panic that I opened a DataFusion issue for in the > hash > > join exec is not caused by the BatchCoalescer, but an overflow. > > > > That said, I think this arrow-rs BatchCoalescer debug_assert needs a > patch > > release before we can ship DF 53 since Comet is having issues with that > too. > > > > On Tue, Mar 17, 2026 at 10:43 AM Andrew Lamb <[email protected]> > wrote: > > > > > The only way really to fix the panic (in debug builds) will be a new > > > version of arrow-rs. I will work on that this week. > > > > > > Should we hold the DataFusion 53 release for that? > > > > > > On Tue, Mar 17, 2026 at 10:28 AM Matt Butrovich <[email protected] > > > > > wrote: > > > > > > > -1 (non-binding) > > > > > > > > CometFuzzTestSuite "join" test causes a panic in the hash join > operator: > > > > https://github.com/apache/datafusion/issues/20995 > > > > > > > > On Tue, Mar 17, 2026 at 8:54 AM Tim Saucer <[email protected]> > wrote: > > > > > > > > > +1 binding > > > > > > > > > > Also verified with `datafusion-python`. > > > > > > > > > > On Tue, Mar 17, 2026 at 7:42 AM Andrew Lamb < > [email protected]> > > > > > wrote: > > > > > > > > > > > +1 (binding) > > > > > > I verified on Mac M3 > > > > > > > > > > > > I believe the verification error is due to a known bug in > arrow-rs[1] > > > > > > > > > > > > Here is a proposed fix to the verification script:[2]. I used the > > > > > modified > > > > > > script and was able to successfully verify the release candidate > > > > > > > > > > > > I will backport the changes to branch-53 for future reference > too > > > > > > > > > > > > Andrew > > > > > > > > > > > > [1]: https://github.com/apache/datafusion/issues/20689 > > > > > > [2]: https://github.com/apache/datafusion/pull/20987 > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Mar 17, 2026 at 3:11 AM Martin Grigorov < > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > Same tests fail here too. > > > > > > > > > > > > > > On Tue, Mar 17, 2026 at 5:42 AM Kevin Liu < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > > > I ran the release script on Mac twice. Getting some failed > tests > > > > both > > > > > > > > times. > > > > > > > > > > > > > > > > Here's the stack trace for the first two: > > > > > > > > ``` > > > > > > > > failures: > > > > > > > > > > > > > > > > ---- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > joins::hash_join::exec::tests::join_full_multi_batch::batch_size_4_2::use_perfect_hash_join_as_possible_1_true > > > > > > > > stdout ---- > > > > > > > > > > > > > > > > thread > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 'joins::hash_join::exec::tests::join_full_multi_batch::batch_size_4_2::use_perfect_hash_join_as_possible_1_true' > > > > > > > > (1910718) panicked at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /var/folders/wq/mxmvfjtd1kjcrm3bdfrn2zd80000gn/T/datafusion-53.0.0.XXXXX.5PoUgcHv4I/apache-datafusion-53.0.0/test-rustup/registry/src/index.crates.io-1949cf8c6b5b557f/arrow-select-58.0.0/src/coalesce/primitive.rs:61:9: > > > > > > > > assertion `left == right` failed > > > > > > > > left: 4 > > > > > > > > right: 2 > > > > > > > > > > > > > > > > ---- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > joins::hash_join::exec::tests::join_full_multi_batch::batch_size_4_2::use_perfect_hash_join_as_possible_2_false > > > > > > > > stdout ---- > > > > > > > > > > > > > > > > thread > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 'joins::hash_join::exec::tests::join_full_multi_batch::batch_size_4_2::use_perfect_hash_join_as_possible_2_false' > > > > > > > > (1910719) panicked at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /var/folders/wq/mxmvfjtd1kjcrm3bdfrn2zd80000gn/T/datafusion-53.0.0.XXXXX.5PoUgcHv4I/apache-datafusion-53.0.0/test-rustup/registry/src/index.crates.io-1949cf8c6b5b557f/arrow-select-58.0.0/src/coalesce/primitive.rs:61:9: > > > > > > > > assertion `left == right` failed > > > > > > > > left: 4 > > > > > > > > right: 2 > > > > > > > > ``` > > > > > > > > > > > > > > > > Best, > > > > > > > > Kevin Liu > > > > > > > > > > > > > > > > On Mon, Mar 16, 2026 at 3:52 PM <[email protected]> wrote: > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > I would like to propose a release of Apache DataFusion > version > > > > > > 53.0.0. > > > > > > > > > This release candidate is based on commit: > > > > > > > > > 28fc91a2a5a8ede5797fde04527f76f351c3cd8b [1] > > > > > > > > > The proposed release tarball and signatures are hosted at > [2]. > > > > > > > > > The changelog is located at [3]. > > > > > > > > > Please download, verify checksums and signatures, run the > unit > > > > > tests, > > > > > > > and > > > > > > > > > vote > > > > > > > > > on the release. The vote will be open for at least 72 > hours. > > > > > > > > > Only votes from PMC members are binding, but all members > of the > > > > > > > community > > > > > > > > > are > > > > > > > > > encouraged to test the release and vote with > "(non-binding)". > > > > > > > > > The standard verification procedure is documented at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/datafusion/blob/main/dev/release/README.md#verifying-release-candidates > > > > > > > > > . > > > > > > > > > [ ] +1 Release this as Apache DataFusion 53.0.0 > > > > > > > > > [ ] +0 > > > > > > > > > [ ] -1 Do not release this as Apache DataFusion 53.0.0 > > > because... > > > > > > > > > Here is my vote: > > > > > > > > > +1 > > > > > > > > > [1]: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/datafusion/tree/28fc91a2a5a8ede5797fde04527f76f351c3cd8b > > > > > > > > > [2]: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-53.0.0-rc2 > > > > > > > > > [3]: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/datafusion/blob/28fc91a2a5a8ede5797fde04527f76f351c3cd8b/CHANGELOG.md > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
