Re: [VOTE] Release Apache DataFusion 46.0.0 RC2
+1 (binding) I ran the verification script on an M3 mac. Thank you so much Xudong! Andrew On Tue, Mar 4, 2025 at 9:47 AM Wang Xudong wrote: > Hi, > > I would like to propose a release of Apache DataFusion version 46.0.0. > > This release candidate is based on commit: > d5ca8307940c1a6345419a2c8d91ef87704659be [1] > The proposed release tarball and signatures are hosted at [2]. > The changelog is located at [3]. > > Please download, verify checksums and signatures, run the unit tests, and > vote > on the release. The vote will be open for at least 72 hours. > > Only votes from PMC members are binding, but all members of the community > are > encouraged to test the release and vote with "(non-binding)". > > The standard verification procedure is documented at > > https://github.com/apache/datafusion/blob/main/dev/release/README.md#verifying-release-candidates > . > > [ ] +1 Release this as Apache DataFusion 46.0.0 > [ ] +0 > [ ] -1 Do not release this as Apache DataFusion 46.0.0 because... > > Here is my vote: > > +1 > > [1]: > > https://github.com/apache/datafusion/tree/d5ca8307940c1a6345419a2c8d91ef87704659be > [2]: > > https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-46.0.0-rc2 > [3]: > > https://github.com/apache/datafusion/blob/d5ca8307940c1a6345419a2c8d91ef87704659be/CHANGELOG.md >
Re: [VOTE] Release Apache DataFusion 46.0.0 RC2
+1 (binding) Verified on M4 Mac. Thanks Xudong. On Tue, Mar 4, 2025 at 7:02 AM Andrew Lamb wrote: > > +1 (binding) > > I ran the verification script on an M3 mac. Thank you so much Xudong! > > Andrew > > On Tue, Mar 4, 2025 at 9:47 AM Wang Xudong wrote: > > > Hi, > > > > I would like to propose a release of Apache DataFusion version 46.0.0. > > > > This release candidate is based on commit: > > d5ca8307940c1a6345419a2c8d91ef87704659be [1] > > The proposed release tarball and signatures are hosted at [2]. > > The changelog is located at [3]. > > > > Please download, verify checksums and signatures, run the unit tests, and > > vote > > on the release. The vote will be open for at least 72 hours. > > > > Only votes from PMC members are binding, but all members of the community > > are > > encouraged to test the release and vote with "(non-binding)". > > > > The standard verification procedure is documented at > > > > https://github.com/apache/datafusion/blob/main/dev/release/README.md#verifying-release-candidates > > . > > > > [ ] +1 Release this as Apache DataFusion 46.0.0 > > [ ] +0 > > [ ] -1 Do not release this as Apache DataFusion 46.0.0 because... > > > > Here is my vote: > > > > +1 > > > > [1]: > > > > https://github.com/apache/datafusion/tree/d5ca8307940c1a6345419a2c8d91ef87704659be > > [2]: > > > > https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-46.0.0-rc2 > > [3]: > > > > https://github.com/apache/datafusion/blob/d5ca8307940c1a6345419a2c8d91ef87704659be/CHANGELOG.md > > - To unsubscribe, e-mail: dev-unsubscr...@datafusion.apache.org For additional commands, e-mail: dev-h...@datafusion.apache.org
[VOTE] Release Apache DataFusion 46.0.0 RC2
Hi, I would like to propose a release of Apache DataFusion version 46.0.0. This release candidate is based on commit: d5ca8307940c1a6345419a2c8d91ef87704659be [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please download, verify checksums and signatures, run the unit tests, and vote on the release. The vote will be open for at least 72 hours. Only votes from PMC members are binding, but all members of the community are encouraged to test the release and vote with "(non-binding)". The standard verification procedure is documented at https://github.com/apache/datafusion/blob/main/dev/release/README.md#verifying-release-candidates . [ ] +1 Release this as Apache DataFusion 46.0.0 [ ] +0 [ ] -1 Do not release this as Apache DataFusion 46.0.0 because... Here is my vote: +1 [1]: https://github.com/apache/datafusion/tree/d5ca8307940c1a6345419a2c8d91ef87704659be [2]: https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-46.0.0-rc2 [3]: https://github.com/apache/datafusion/blob/d5ca8307940c1a6345419a2c8d91ef87704659be/CHANGELOG.md
Re: [VOTE] Release Apache DataFusion 46.0.0 RC2
Oops. Thanks, Xudong!
On Tue, Mar 4, 2025 at 3:38 PM Andy Grove wrote:
> +1 (binding)
>
> Tested with verification script and with Comet upgrade.
>
> Thanks, Andrew!
>
>
> On Tue, Mar 4, 2025 at 11:22 AM Kevin Liu wrote:
>
>> Thank you, that fixed the issue.
>>
>> +1 (non-binding).
>> Verified by running `./dev/release/verify-release-candidate.sh 46.0.0 2`
>> on
>> M1 Mac.
>>
>> Best,
>> Kevin Liu
>>
>>
>> On Tue, Mar 4, 2025 at 8:48 AM Andrew Lamb wrote:
>>
>> > I think you need to set the limit of files higher. Something like the
>> > following
>> >
>> > ```
>> > ulimit -n 1
>> > ```
>> >
>> > We discussed this on a ticket previously[1]. Maybe we should fix it for
>> > real :thinking:
>> >
>> > [1] https://github.com/apache/datafusion/issues/10674
>> >
>> >
>> >
>> > On Tue, Mar 4, 2025 at 11:30 AM Kevin Liu
>> wrote:
>> >
>> > > Hey folks,
>> > >
>> > > I'm having issues with one particular test. It could be my local
>> > machine. I
>> > > tried restarting my laptop and running the test in isolation (`cargo
>> test
>> > > -p datafusion --test fuzz`) but the error still persists.
>> > >
>> > > ```
>> > > ./dev/release/verify-release-candidate.sh 46.0.0 2
>> > >
>> > > failures:
>> > >
>> > > fuzz_cases::sort_fuzz::test_sort_10k_mem stdout
>> > >
>> > > thread 'fuzz_cases::sort_fuzz::test_sort_10k_mem' panicked at
>> > > datafusion/core/tests/fuzz_cases/sort_fuzz.rs:272:63:
>> > > called `Result::unwrap()` on an `Err` value: Execution("Failed to
>> create
>> > > partition file at
>> > >
>> >
>> \"/var/folders/f1/3_vzsn7x1jq9hszb3z9y6f0mgn/T/.tmpYjVAvj/.tmph79STL\":
>> > > Os { code: 24, kind: Uncategorized, message: \"Too many open files\"
>> }")
>> > > note: run with `RUST_BACKTRACE=1` environment variable to display a
>> > > backtrace
>> > >
>> > >
>> > > failures:
>> > > fuzz_cases::sort_fuzz::test_sort_10k_mem
>> > >
>> > > test result: FAILED. 64 passed; 1 failed; 0 ignored; 0 measured; 0
>> > filtered
>> > > out; finished in 74.07s
>> > > ```
>> > >
>> > > Best,
>> > > Kevin Liu
>> > >
>> > > On Tue, Mar 4, 2025 at 7:21 AM L. C. Hsieh wrote:
>> > >
>> > > > +1 (binding)
>> > > >
>> > > > Verified on M4 Mac.
>> > > >
>> > > > Thanks Xudong.
>> > > >
>> > > > On Tue, Mar 4, 2025 at 7:02 AM Andrew Lamb
>> > wrote:
>> > > > >
>> > > > > +1 (binding)
>> > > > >
>> > > > > I ran the verification script on an M3 mac. Thank you so much
>> Xudong!
>> > > > >
>> > > > > Andrew
>> > > > >
>> > > > > On Tue, Mar 4, 2025 at 9:47 AM Wang Xudong <
>> wxd963996...@gmail.com>
>> > > > wrote:
>> > > > >
>> > > > > > Hi,
>> > > > > >
>> > > > > > I would like to propose a release of Apache DataFusion version
>> > > 46.0.0.
>> > > > > >
>> > > > > > This release candidate is based on commit:
>> > > > > > d5ca8307940c1a6345419a2c8d91ef87704659be [1]
>> > > > > > The proposed release tarball and signatures are hosted at [2].
>> > > > > > The changelog is located at [3].
>> > > > > >
>> > > > > > Please download, verify checksums and signatures, run the unit
>> > tests,
>> > > > and
>> > > > > > vote
>> > > > > > on the release. The vote will be open for at least 72 hours.
>> > > > > >
>> > > > > > Only votes from PMC members are binding, but all members of the
>> > > > community
>> > > > > > are
>> > > > > > encouraged to test the release and vote with "(non-binding)".
>> > > > > >
>> > > > > > The standard verification procedure is documented at
>> > > > > >
>> > > > > >
>> > > >
>> > >
>> >
>> https://github.com/apache/datafusion/blob/main/dev/release/README.md#verifying-release-candidates
>> > > > > > .
>> > > > > >
>> > > > > > [ ] +1 Release this as Apache DataFusion 46.0.0
>> > > > > > [ ] +0
>> > > > > > [ ] -1 Do not release this as Apache DataFusion 46.0.0
>> because...
>> > > > > >
>> > > > > > Here is my vote:
>> > > > > >
>> > > > > > +1
>> > > > > >
>> > > > > > [1]:
>> > > > > >
>> > > > > >
>> > > >
>> > >
>> >
>> https://github.com/apache/datafusion/tree/d5ca8307940c1a6345419a2c8d91ef87704659be
>> > > > > > [2]:
>> > > > > >
>> > > > > >
>> > > >
>> > >
>> >
>> https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-46.0.0-rc2
>> > > > > > [3]:
>> > > > > >
>> > > > > >
>> > > >
>> > >
>> >
>> https://github.com/apache/datafusion/blob/d5ca8307940c1a6345419a2c8d91ef87704659be/CHANGELOG.md
>> > > > > >
>> > > >
>> > > >
>> -
>> > > > To unsubscribe, e-mail: dev-unsubscr...@datafusion.apache.org
>> > > > For additional commands, e-mail: dev-h...@datafusion.apache.org
>> > > >
>> > > >
>> > >
>> >
>>
>
Re: [VOTE] Release Apache DataFusion 46.0.0 RC2
+1 (binding)
Tested with verification script and with Comet upgrade.
Thanks, Andrew!
On Tue, Mar 4, 2025 at 11:22 AM Kevin Liu wrote:
> Thank you, that fixed the issue.
>
> +1 (non-binding).
> Verified by running `./dev/release/verify-release-candidate.sh 46.0.0 2` on
> M1 Mac.
>
> Best,
> Kevin Liu
>
>
> On Tue, Mar 4, 2025 at 8:48 AM Andrew Lamb wrote:
>
> > I think you need to set the limit of files higher. Something like the
> > following
> >
> > ```
> > ulimit -n 1
> > ```
> >
> > We discussed this on a ticket previously[1]. Maybe we should fix it for
> > real :thinking:
> >
> > [1] https://github.com/apache/datafusion/issues/10674
> >
> >
> >
> > On Tue, Mar 4, 2025 at 11:30 AM Kevin Liu wrote:
> >
> > > Hey folks,
> > >
> > > I'm having issues with one particular test. It could be my local
> > machine. I
> > > tried restarting my laptop and running the test in isolation (`cargo
> test
> > > -p datafusion --test fuzz`) but the error still persists.
> > >
> > > ```
> > > ./dev/release/verify-release-candidate.sh 46.0.0 2
> > >
> > > failures:
> > >
> > > fuzz_cases::sort_fuzz::test_sort_10k_mem stdout
> > >
> > > thread 'fuzz_cases::sort_fuzz::test_sort_10k_mem' panicked at
> > > datafusion/core/tests/fuzz_cases/sort_fuzz.rs:272:63:
> > > called `Result::unwrap()` on an `Err` value: Execution("Failed to
> create
> > > partition file at
> > >
> >
> \"/var/folders/f1/3_vzsn7x1jq9hszb3z9y6f0mgn/T/.tmpYjVAvj/.tmph79STL\":
> > > Os { code: 24, kind: Uncategorized, message: \"Too many open files\"
> }")
> > > note: run with `RUST_BACKTRACE=1` environment variable to display a
> > > backtrace
> > >
> > >
> > > failures:
> > > fuzz_cases::sort_fuzz::test_sort_10k_mem
> > >
> > > test result: FAILED. 64 passed; 1 failed; 0 ignored; 0 measured; 0
> > filtered
> > > out; finished in 74.07s
> > > ```
> > >
> > > Best,
> > > Kevin Liu
> > >
> > > On Tue, Mar 4, 2025 at 7:21 AM L. C. Hsieh wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > Verified on M4 Mac.
> > > >
> > > > Thanks Xudong.
> > > >
> > > > On Tue, Mar 4, 2025 at 7:02 AM Andrew Lamb
> > wrote:
> > > > >
> > > > > +1 (binding)
> > > > >
> > > > > I ran the verification script on an M3 mac. Thank you so much
> Xudong!
> > > > >
> > > > > Andrew
> > > > >
> > > > > On Tue, Mar 4, 2025 at 9:47 AM Wang Xudong >
> > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I would like to propose a release of Apache DataFusion version
> > > 46.0.0.
> > > > > >
> > > > > > This release candidate is based on commit:
> > > > > > d5ca8307940c1a6345419a2c8d91ef87704659be [1]
> > > > > > The proposed release tarball and signatures are hosted at [2].
> > > > > > The changelog is located at [3].
> > > > > >
> > > > > > Please download, verify checksums and signatures, run the unit
> > tests,
> > > > and
> > > > > > vote
> > > > > > on the release. The vote will be open for at least 72 hours.
> > > > > >
> > > > > > Only votes from PMC members are binding, but all members of the
> > > > community
> > > > > > are
> > > > > > encouraged to test the release and vote with "(non-binding)".
> > > > > >
> > > > > > The standard verification procedure is documented at
> > > > > >
> > > > > >
> > > >
> > >
> >
> https://github.com/apache/datafusion/blob/main/dev/release/README.md#verifying-release-candidates
> > > > > > .
> > > > > >
> > > > > > [ ] +1 Release this as Apache DataFusion 46.0.0
> > > > > > [ ] +0
> > > > > > [ ] -1 Do not release this as Apache DataFusion 46.0.0 because...
> > > > > >
> > > > > > Here is my vote:
> > > > > >
> > > > > > +1
> > > > > >
> > > > > > [1]:
> > > > > >
> > > > > >
> > > >
> > >
> >
> https://github.com/apache/datafusion/tree/d5ca8307940c1a6345419a2c8d91ef87704659be
> > > > > > [2]:
> > > > > >
> > > > > >
> > > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-46.0.0-rc2
> > > > > > [3]:
> > > > > >
> > > > > >
> > > >
> > >
> >
> https://github.com/apache/datafusion/blob/d5ca8307940c1a6345419a2c8d91ef87704659be/CHANGELOG.md
> > > > > >
> > > >
> > > > -
> > > > To unsubscribe, e-mail: dev-unsubscr...@datafusion.apache.org
> > > > For additional commands, e-mail: dev-h...@datafusion.apache.org
> > > >
> > > >
> > >
> >
>
Re: [VOTE] Release Apache DataFusion 46.0.0 RC1
Update: we have merged a fix for the scripts [1] I suggest we make a new RC with the fix and start a new voting thread [1]: https://github.com/apache/datafusion/pull/14990 On Mon, Mar 3, 2025 at 12:15 PM Andy Grove wrote: > I have a PR that may fix the issue: > https://github.com/apache/datafusion/pull/14985 > > On Mon, Mar 3, 2025 at 10:04 AM Andrew Lamb > wrote: > > > I think something in rustup changed: > > > https://github.com/apache/datafusion/issues/14982#issuecomment-2694821546 > > > > On Mon, Mar 3, 2025 at 11:58 AM Andy Grove > wrote: > > > > > I am seeing similar errors today on all Comet PRs. Something seems to > > have > > > changed with the 1.85 release. For Comet, changing the rust-toolchain > > file > > > to refer to channel "stable" rather than a specific version number > seems > > to > > > fix this. > > > > > > On Mon, Mar 3, 2025 at 9:25 AM Andrew Lamb > wrote: > > > > > > > I have a PR with a workaround for the release verification issue[1]. > > > > > > > > However, I am not sure if there is a problem with the script or if > the > > > > issue is with rustup. > > > > > > > > Perhaps for now to unblock the release, we can merge [1] and make RC2 > > > > > > > > Andrew > > > > > > > > [1] https://github.com/apache/datafusion/pull/14983 > > > > > > > > On Mon, Mar 3, 2025 at 10:44 AM Andrew Lamb > > > > wrote: > > > > > > > > > Thanks Xudong! > > > > > > > > > > I ran the verification script > > > > > ``` > > > > > $ ./dev/release/verify-release-candidate.sh 46.0.0 1 > > > > > ``` > > > > > > > > > > And it errored in the following way: > > > > > > > > > > ``` > > > > > error: toolchain '1.85.0-aarch64-apple-darwin' is not installed > > > > > help: run `rustup toolchain install 1.85.0-aarch64-apple-darwin` to > > > > install > > > > > it > > > > > + cleanup > > > > > + '[' no = yes ']' > > > > > + echo 'Failed to verify release candidate. See > > > > > > > > > > > > > > > > > > > > /var/folders/1l/tg68jc6550gg8xqf1hr4mlwrgn/T/datafusion-46.0.0.X.ps1ExCTVRs > > > > > for details.' > > > > > Failed to verify release candidate. See > > > > > > > > > > > > > > > > > > > > /var/folders/1l/tg68jc6550gg8xqf1hr4mlwrgn/T/datafusion-46.0.0.X.ps1ExCTVRs > > > > > for details. > > > > > ``` > > > > > > > > > > It is not clear to me what is going on and this seems to have > started > > > > > happening on CI on main too. > > > > > > > > > > I filed a ticket to track[1] > > > > > > > > > > [1]: https://github.com/apache/datafusion/issues/14982 > > > > > > > > > > On Mon, Mar 3, 2025 at 8:44 AM Wang Xudong > > > > > wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > I would like to propose a release of Apache DataFusion version > > > 46.0.0. > > > > > > > > > > > > This release candidate is based on commit: > > > > > > ec4862fa2d870fcab973fd1589ef99b6bf8d560f [1] > > > > > > The proposed release tarball and signatures are hosted at [2]. > > > > > > The changelog is located at [3]. > > > > > > > > > > > > Please download, verify checksums and signatures, run the unit > > tests, > > > > and > > > > > > vote > > > > > > on the release. The vote will be open for at least 72 hours. > > > > > > > > > > > > Only votes from PMC members are binding, but all members of the > > > > community > > > > > > are > > > > > > encouraged to test the release and vote with "(non-binding)". > > > > > > > > > > > > The standard verification procedure is documented at > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/datafusion/blob/main/dev/release/README.md#verifying-release-candidates > > > > > > . > > > > > > > > > > > > [ ] +1 Release this as Apache DataFusion 46.0.0 > > > > > > [ ] +0 > > > > > > [ ] -1 Do not release this as Apache DataFusion 46.0.0 because... > > > > > > > > > > > > Here is my vote: > > > > > > > > > > > > +1 (non binding) > > > > > > > > > > > > [1]: > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/datafusion/tree/ec4862fa2d870fcab973fd1589ef99b6bf8d560f > > > > > > [2]: > > > > > > > > > > > > > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-46.0.0-rc1 > > > > > > [3]: > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/datafusion/blob/ec4862fa2d870fcab973fd1589ef99b6bf8d560f/CHANGELOG.md > > > > > > > > > > > > > > > > > > > > >
Re: [VOTE] Release Apache DataFusion 46.0.0 RC2
I think you need to set the limit of files higher. Something like the
following
```
ulimit -n 1
```
We discussed this on a ticket previously[1]. Maybe we should fix it for
real :thinking:
[1] https://github.com/apache/datafusion/issues/10674
On Tue, Mar 4, 2025 at 11:30 AM Kevin Liu wrote:
> Hey folks,
>
> I'm having issues with one particular test. It could be my local machine. I
> tried restarting my laptop and running the test in isolation (`cargo test
> -p datafusion --test fuzz`) but the error still persists.
>
> ```
> ./dev/release/verify-release-candidate.sh 46.0.0 2
>
> failures:
>
> fuzz_cases::sort_fuzz::test_sort_10k_mem stdout
>
> thread 'fuzz_cases::sort_fuzz::test_sort_10k_mem' panicked at
> datafusion/core/tests/fuzz_cases/sort_fuzz.rs:272:63:
> called `Result::unwrap()` on an `Err` value: Execution("Failed to create
> partition file at
> \"/var/folders/f1/3_vzsn7x1jq9hszb3z9y6f0mgn/T/.tmpYjVAvj/.tmph79STL\":
> Os { code: 24, kind: Uncategorized, message: \"Too many open files\" }")
> note: run with `RUST_BACKTRACE=1` environment variable to display a
> backtrace
>
>
> failures:
> fuzz_cases::sort_fuzz::test_sort_10k_mem
>
> test result: FAILED. 64 passed; 1 failed; 0 ignored; 0 measured; 0 filtered
> out; finished in 74.07s
> ```
>
> Best,
> Kevin Liu
>
> On Tue, Mar 4, 2025 at 7:21 AM L. C. Hsieh wrote:
>
> > +1 (binding)
> >
> > Verified on M4 Mac.
> >
> > Thanks Xudong.
> >
> > On Tue, Mar 4, 2025 at 7:02 AM Andrew Lamb wrote:
> > >
> > > +1 (binding)
> > >
> > > I ran the verification script on an M3 mac. Thank you so much Xudong!
> > >
> > > Andrew
> > >
> > > On Tue, Mar 4, 2025 at 9:47 AM Wang Xudong
> > wrote:
> > >
> > > > Hi,
> > > >
> > > > I would like to propose a release of Apache DataFusion version
> 46.0.0.
> > > >
> > > > This release candidate is based on commit:
> > > > d5ca8307940c1a6345419a2c8d91ef87704659be [1]
> > > > The proposed release tarball and signatures are hosted at [2].
> > > > The changelog is located at [3].
> > > >
> > > > Please download, verify checksums and signatures, run the unit tests,
> > and
> > > > vote
> > > > on the release. The vote will be open for at least 72 hours.
> > > >
> > > > Only votes from PMC members are binding, but all members of the
> > community
> > > > are
> > > > encouraged to test the release and vote with "(non-binding)".
> > > >
> > > > The standard verification procedure is documented at
> > > >
> > > >
> >
> https://github.com/apache/datafusion/blob/main/dev/release/README.md#verifying-release-candidates
> > > > .
> > > >
> > > > [ ] +1 Release this as Apache DataFusion 46.0.0
> > > > [ ] +0
> > > > [ ] -1 Do not release this as Apache DataFusion 46.0.0 because...
> > > >
> > > > Here is my vote:
> > > >
> > > > +1
> > > >
> > > > [1]:
> > > >
> > > >
> >
> https://github.com/apache/datafusion/tree/d5ca8307940c1a6345419a2c8d91ef87704659be
> > > > [2]:
> > > >
> > > >
> >
> https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-46.0.0-rc2
> > > > [3]:
> > > >
> > > >
> >
> https://github.com/apache/datafusion/blob/d5ca8307940c1a6345419a2c8d91ef87704659be/CHANGELOG.md
> > > >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@datafusion.apache.org
> > For additional commands, e-mail: dev-h...@datafusion.apache.org
> >
> >
>
Re: [VOTE] Release Apache DataFusion 46.0.0 RC2
Thank you, that fixed the issue.
+1 (non-binding).
Verified by running `./dev/release/verify-release-candidate.sh 46.0.0 2` on
M1 Mac.
Best,
Kevin Liu
On Tue, Mar 4, 2025 at 8:48 AM Andrew Lamb wrote:
> I think you need to set the limit of files higher. Something like the
> following
>
> ```
> ulimit -n 1
> ```
>
> We discussed this on a ticket previously[1]. Maybe we should fix it for
> real :thinking:
>
> [1] https://github.com/apache/datafusion/issues/10674
>
>
>
> On Tue, Mar 4, 2025 at 11:30 AM Kevin Liu wrote:
>
> > Hey folks,
> >
> > I'm having issues with one particular test. It could be my local
> machine. I
> > tried restarting my laptop and running the test in isolation (`cargo test
> > -p datafusion --test fuzz`) but the error still persists.
> >
> > ```
> > ./dev/release/verify-release-candidate.sh 46.0.0 2
> >
> > failures:
> >
> > fuzz_cases::sort_fuzz::test_sort_10k_mem stdout
> >
> > thread 'fuzz_cases::sort_fuzz::test_sort_10k_mem' panicked at
> > datafusion/core/tests/fuzz_cases/sort_fuzz.rs:272:63:
> > called `Result::unwrap()` on an `Err` value: Execution("Failed to create
> > partition file at
> >
> \"/var/folders/f1/3_vzsn7x1jq9hszb3z9y6f0mgn/T/.tmpYjVAvj/.tmph79STL\":
> > Os { code: 24, kind: Uncategorized, message: \"Too many open files\" }")
> > note: run with `RUST_BACKTRACE=1` environment variable to display a
> > backtrace
> >
> >
> > failures:
> > fuzz_cases::sort_fuzz::test_sort_10k_mem
> >
> > test result: FAILED. 64 passed; 1 failed; 0 ignored; 0 measured; 0
> filtered
> > out; finished in 74.07s
> > ```
> >
> > Best,
> > Kevin Liu
> >
> > On Tue, Mar 4, 2025 at 7:21 AM L. C. Hsieh wrote:
> >
> > > +1 (binding)
> > >
> > > Verified on M4 Mac.
> > >
> > > Thanks Xudong.
> > >
> > > On Tue, Mar 4, 2025 at 7:02 AM Andrew Lamb
> wrote:
> > > >
> > > > +1 (binding)
> > > >
> > > > I ran the verification script on an M3 mac. Thank you so much Xudong!
> > > >
> > > > Andrew
> > > >
> > > > On Tue, Mar 4, 2025 at 9:47 AM Wang Xudong
> > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I would like to propose a release of Apache DataFusion version
> > 46.0.0.
> > > > >
> > > > > This release candidate is based on commit:
> > > > > d5ca8307940c1a6345419a2c8d91ef87704659be [1]
> > > > > The proposed release tarball and signatures are hosted at [2].
> > > > > The changelog is located at [3].
> > > > >
> > > > > Please download, verify checksums and signatures, run the unit
> tests,
> > > and
> > > > > vote
> > > > > on the release. The vote will be open for at least 72 hours.
> > > > >
> > > > > Only votes from PMC members are binding, but all members of the
> > > community
> > > > > are
> > > > > encouraged to test the release and vote with "(non-binding)".
> > > > >
> > > > > The standard verification procedure is documented at
> > > > >
> > > > >
> > >
> >
> https://github.com/apache/datafusion/blob/main/dev/release/README.md#verifying-release-candidates
> > > > > .
> > > > >
> > > > > [ ] +1 Release this as Apache DataFusion 46.0.0
> > > > > [ ] +0
> > > > > [ ] -1 Do not release this as Apache DataFusion 46.0.0 because...
> > > > >
> > > > > Here is my vote:
> > > > >
> > > > > +1
> > > > >
> > > > > [1]:
> > > > >
> > > > >
> > >
> >
> https://github.com/apache/datafusion/tree/d5ca8307940c1a6345419a2c8d91ef87704659be
> > > > > [2]:
> > > > >
> > > > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-46.0.0-rc2
> > > > > [3]:
> > > > >
> > > > >
> > >
> >
> https://github.com/apache/datafusion/blob/d5ca8307940c1a6345419a2c8d91ef87704659be/CHANGELOG.md
> > > > >
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@datafusion.apache.org
> > > For additional commands, e-mail: dev-h...@datafusion.apache.org
> > >
> > >
> >
>
Re: [VOTE] Release Apache DataFusion Ballista 44.0.0 RC1
The vote passes with five +1 votes (three binding). Thanks, everyone. I have published the source release and published the crates. Andy. On Sat, Mar 1, 2025 at 2:03 PM Andrew Lamb wrote: > +1 (binding) > > Verified on M3 Mac > > Looks like a good release. Thanks Marko for all the work on this one and > Andy for doing the release process! > > Andrew > > On Sat, Mar 1, 2025 at 2:40 PM Marko Milenković > wrote: > > > +1 non-binding > > Verified on Mac M4 > > > > Thanks Andy! > > > > Best regards, > > > > -- > > Marko Milenković > > > > > > > > > > > On 1 Mar 2025, at 15:26, Andy Grove wrote: > > > > > > Hi, > > > > > > I would like to propose a release of Apache DataFusion Ballista version > > > 44.0.0. > > > > > > This release candidate is based on commit: > > > fc5a99c2e9fb52ae06127fa0fe2f19d6ff9406a6 [1] > > > The proposed release tarball and signatures are hosted at [2]. > > > The changelog is located at [3]. > > > > > > Please download, verify checksums and signatures, run the unit tests, > and > > > vote > > > on the release. The vote will be open for at least 72 hours. > > > > > > Only votes from PMC members are binding, but all members of the > community > > > are > > > encouraged to test the release and vote with "(non-binding)". > > > > > > The standard verification procedure is documented at > > > > > > https://github.com/apache/datafusion-ballista/blob/main/dev/release/README.md#verifying-release-candidates > > > . > > > > > > [ ] +1 Release this as Apache DataFusion Ballista 44.0.0 > > > [ ] +0 > > > [ ] -1 Do not release this as Apache DataFusion Ballista 44.0.0 > > because... > > > > > > My vote is +1 > > > > > > [1]: > > > > > > https://github.com/apache/datafusion-ballista/tree/fc5a99c2e9fb52ae06127fa0fe2f19d6ff9406a6 > > > [2]: > > > > > > https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-ballista-44.0.0-rc1 > > > [3]: > > > > > > https://github.com/apache/datafusion-ballista/blob/fc5a99c2e9fb52ae06127fa0fe2f19d6ff9406a6/CHANGELOG.md > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@datafusion.apache.org > > For additional commands, e-mail: dev-h...@datafusion.apache.org > > > > >
Re: [VOTE] Release Apache DataFusion 46.0.0 RC2
+1 (binding)
Verified with AMD64 ArchLinux.
Thanks all!
On Tue, Mar 4, 2025 at 2:40 PM Andy Grove wrote:
> Oops. Thanks, Xudong!
>
> On Tue, Mar 4, 2025 at 3:38 PM Andy Grove wrote:
>
> > +1 (binding)
> >
> > Tested with verification script and with Comet upgrade.
> >
> > Thanks, Andrew!
> >
> >
> > On Tue, Mar 4, 2025 at 11:22 AM Kevin Liu wrote:
> >
> >> Thank you, that fixed the issue.
> >>
> >> +1 (non-binding).
> >> Verified by running `./dev/release/verify-release-candidate.sh 46.0.0 2`
> >> on
> >> M1 Mac.
> >>
> >> Best,
> >> Kevin Liu
> >>
> >>
> >> On Tue, Mar 4, 2025 at 8:48 AM Andrew Lamb
> wrote:
> >>
> >> > I think you need to set the limit of files higher. Something like the
> >> > following
> >> >
> >> > ```
> >> > ulimit -n 1
> >> > ```
> >> >
> >> > We discussed this on a ticket previously[1]. Maybe we should fix it
> for
> >> > real :thinking:
> >> >
> >> > [1] https://github.com/apache/datafusion/issues/10674
> >> >
> >> >
> >> >
> >> > On Tue, Mar 4, 2025 at 11:30 AM Kevin Liu
> >> wrote:
> >> >
> >> > > Hey folks,
> >> > >
> >> > > I'm having issues with one particular test. It could be my local
> >> > machine. I
> >> > > tried restarting my laptop and running the test in isolation (`cargo
> >> test
> >> > > -p datafusion --test fuzz`) but the error still persists.
> >> > >
> >> > > ```
> >> > > ./dev/release/verify-release-candidate.sh 46.0.0 2
> >> > >
> >> > > failures:
> >> > >
> >> > > fuzz_cases::sort_fuzz::test_sort_10k_mem stdout
> >> > >
> >> > > thread 'fuzz_cases::sort_fuzz::test_sort_10k_mem' panicked at
> >> > > datafusion/core/tests/fuzz_cases/sort_fuzz.rs:272:63:
> >> > > called `Result::unwrap()` on an `Err` value: Execution("Failed to
> >> create
> >> > > partition file at
> >> > >
> >> >
> >>
> \"/var/folders/f1/3_vzsn7x1jq9hszb3z9y6f0mgn/T/.tmpYjVAvj/.tmph79STL\":
> >> > > Os { code: 24, kind: Uncategorized, message: \"Too many open files\"
> >> }")
> >> > > note: run with `RUST_BACKTRACE=1` environment variable to display a
> >> > > backtrace
> >> > >
> >> > >
> >> > > failures:
> >> > > fuzz_cases::sort_fuzz::test_sort_10k_mem
> >> > >
> >> > > test result: FAILED. 64 passed; 1 failed; 0 ignored; 0 measured; 0
> >> > filtered
> >> > > out; finished in 74.07s
> >> > > ```
> >> > >
> >> > > Best,
> >> > > Kevin Liu
> >> > >
> >> > > On Tue, Mar 4, 2025 at 7:21 AM L. C. Hsieh
> wrote:
> >> > >
> >> > > > +1 (binding)
> >> > > >
> >> > > > Verified on M4 Mac.
> >> > > >
> >> > > > Thanks Xudong.
> >> > > >
> >> > > > On Tue, Mar 4, 2025 at 7:02 AM Andrew Lamb
> >> > wrote:
> >> > > > >
> >> > > > > +1 (binding)
> >> > > > >
> >> > > > > I ran the verification script on an M3 mac. Thank you so much
> >> Xudong!
> >> > > > >
> >> > > > > Andrew
> >> > > > >
> >> > > > > On Tue, Mar 4, 2025 at 9:47 AM Wang Xudong <
> >> wxd963996...@gmail.com>
> >> > > > wrote:
> >> > > > >
> >> > > > > > Hi,
> >> > > > > >
> >> > > > > > I would like to propose a release of Apache DataFusion version
> >> > > 46.0.0.
> >> > > > > >
> >> > > > > > This release candidate is based on commit:
> >> > > > > > d5ca8307940c1a6345419a2c8d91ef87704659be [1]
> >> > > > > > The proposed release tarball and signatures are hosted at [2].
> >> > > > > > The changelog is located at [3].
> >> > > > > >
> >> > > > > > Please download, verify checksums and signatures, run the unit
> >> > tests,
> >> > > > and
> >> > > > > > vote
> >> > > > > > on the release. The vote will be open for at least 72 hours.
> >> > > > > >
> >> > > > > > Only votes from PMC members are binding, but all members of
> the
> >> > > > community
> >> > > > > > are
> >> > > > > > encouraged to test the release and vote with "(non-binding)".
> >> > > > > >
> >> > > > > > The standard verification procedure is documented at
> >> > > > > >
> >> > > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/apache/datafusion/blob/main/dev/release/README.md#verifying-release-candidates
> >> > > > > > .
> >> > > > > >
> >> > > > > > [ ] +1 Release this as Apache DataFusion 46.0.0
> >> > > > > > [ ] +0
> >> > > > > > [ ] -1 Do not release this as Apache DataFusion 46.0.0
> >> because...
> >> > > > > >
> >> > > > > > Here is my vote:
> >> > > > > >
> >> > > > > > +1
> >> > > > > >
> >> > > > > > [1]:
> >> > > > > >
> >> > > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/apache/datafusion/tree/d5ca8307940c1a6345419a2c8d91ef87704659be
> >> > > > > > [2]:
> >> > > > > >
> >> > > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-46.0.0-rc2
> >> > > > > > [3]:
> >> > > > > >
> >> > > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/apache/datafusion/blob/d5ca8307940c1a6345419a2c8d91ef87704659be/CHANGELOG.md
> >> > > > > >
> >> > > >
> >> > > >
> >> -
> >> > > > To unsubscribe, e-mail: dev-unsubscr...@datafusion.apache.org
> >> > > > For additional commands, e-mail: dev-
Re: [VOTE] Release Apache DataFusion 46.0.0 RC2
Hey folks,
I'm having issues with one particular test. It could be my local machine. I
tried restarting my laptop and running the test in isolation (`cargo test
-p datafusion --test fuzz`) but the error still persists.
```
./dev/release/verify-release-candidate.sh 46.0.0 2
failures:
fuzz_cases::sort_fuzz::test_sort_10k_mem stdout
thread 'fuzz_cases::sort_fuzz::test_sort_10k_mem' panicked at
datafusion/core/tests/fuzz_cases/sort_fuzz.rs:272:63:
called `Result::unwrap()` on an `Err` value: Execution("Failed to create
partition file at
\"/var/folders/f1/3_vzsn7x1jq9hszb3z9y6f0mgn/T/.tmpYjVAvj/.tmph79STL\":
Os { code: 24, kind: Uncategorized, message: \"Too many open files\" }")
note: run with `RUST_BACKTRACE=1` environment variable to display a
backtrace
failures:
fuzz_cases::sort_fuzz::test_sort_10k_mem
test result: FAILED. 64 passed; 1 failed; 0 ignored; 0 measured; 0 filtered
out; finished in 74.07s
```
Best,
Kevin Liu
On Tue, Mar 4, 2025 at 7:21 AM L. C. Hsieh wrote:
> +1 (binding)
>
> Verified on M4 Mac.
>
> Thanks Xudong.
>
> On Tue, Mar 4, 2025 at 7:02 AM Andrew Lamb wrote:
> >
> > +1 (binding)
> >
> > I ran the verification script on an M3 mac. Thank you so much Xudong!
> >
> > Andrew
> >
> > On Tue, Mar 4, 2025 at 9:47 AM Wang Xudong
> wrote:
> >
> > > Hi,
> > >
> > > I would like to propose a release of Apache DataFusion version 46.0.0.
> > >
> > > This release candidate is based on commit:
> > > d5ca8307940c1a6345419a2c8d91ef87704659be [1]
> > > The proposed release tarball and signatures are hosted at [2].
> > > The changelog is located at [3].
> > >
> > > Please download, verify checksums and signatures, run the unit tests,
> and
> > > vote
> > > on the release. The vote will be open for at least 72 hours.
> > >
> > > Only votes from PMC members are binding, but all members of the
> community
> > > are
> > > encouraged to test the release and vote with "(non-binding)".
> > >
> > > The standard verification procedure is documented at
> > >
> > >
> https://github.com/apache/datafusion/blob/main/dev/release/README.md#verifying-release-candidates
> > > .
> > >
> > > [ ] +1 Release this as Apache DataFusion 46.0.0
> > > [ ] +0
> > > [ ] -1 Do not release this as Apache DataFusion 46.0.0 because...
> > >
> > > Here is my vote:
> > >
> > > +1
> > >
> > > [1]:
> > >
> > >
> https://github.com/apache/datafusion/tree/d5ca8307940c1a6345419a2c8d91ef87704659be
> > > [2]:
> > >
> > >
> https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-46.0.0-rc2
> > > [3]:
> > >
> > >
> https://github.com/apache/datafusion/blob/d5ca8307940c1a6345419a2c8d91ef87704659be/CHANGELOG.md
> > >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@datafusion.apache.org
> For additional commands, e-mail: dev-h...@datafusion.apache.org
>
>
