Re: Proposal for new datafusion-benchmarks repository

[Parth Chandra]

Great idea Andy.
I couldn't see any of the feedback, btw, nor could I add any comments in
the doc :(

Anyway here are some similar efforts (Big data/SQL benchmarks) from other
communities -
Trino: https://github.com/trinodb/tpcds, https://github.com/trinodb/tpch
Spark - https://github.com/databricks/spark-sql-perf
AmpLab - https://amplab.cs.berkeley.edu/benchmark/

And some interesting reads -
https://www.cs.cmu.edu/~pavlo/papers/benchmarks-sigmod09.pdf
https://www.vldb.org/pvldb/vol13/p3285-gruenheid.pdf


On Tue, May 14, 2024 at 6:16 AM Andy Grove 
wrote:

> Thank you for the feedback on the proposal, which has all been positive.
>
> I have now created the repository, and I plan on creating some PRs this
> week to add some initial documentation and scripts.
>
> [image: datafusion-benchmarks.png]
>
> apache/datafusion-benchmarks: Apache DataFusion Benchmarks
> 
> github.com 
> 
>
>
> Thanks,
>
> Andy.
>
>
>
> On May 12, 2024, at 8:54 AM, Andy Grove 
> wrote:
>
> Hello,
>
> I would like to propose creating a new datafusion-benchmarks repository
> for shared documentation and scripts that can help with benchmarking
> efforts across DataFusion and its subprojects. Please let me know your
> thoughts in the attached Google document.
>
> Thanks,
>
> Andy.
>
>
>
> 
>
> DataFusion Benchmarking Repository Proposal
> 
> docs.google.com
> 
>
> 
>
>
>
>

Re: [VOTE] Release Apache DataFusion Comet 0.1.0 RC2

[Parth Chandra]

Src tarball verified successfully.

However, building the artifacts from src gives an error because the
makefile does not have "-Dmaven.gitcommitid.skip=true" specified.
$ make release ( or just make)
...
[ERROR] Failed to execute goal
io.github.git-commit-id:git-commit-id-maven-plugin:4.9.9:revision
(get-the-git-infos) on project comet-common-spark3.4_2.12: .git directory
is not found! Please specify a valid [dotGitDirectory] in your pom.xml ->
[Help 1]


On Sat, Jul 20, 2024 at 1:56 PM Andy Grove 
wrote:

> Hi,
>
> I would like to propose a release of Apache DataFusion Comet version 0.1.0.
>
> This release candidate is based on commit:
> df980bf67e59f439521b90dd2139c6a8d706c0bc [1]
> The proposed release tarball and signatures are hosted at [2].
> The changelog is located at [3].
>
> Please download, verify checksums and signatures, run the unit tests, and
> vote
> on the release. The vote will be open for at least 72 hours.
>
> Only votes from PMC members are binding, but all members of the community
> are
> encouraged to test the release and vote with "(non-binding)".
>
> The standard verification procedure is documented at
> https://github.com/apache/datafusion-comet/blob/main/dev/release/verifying-release-candidates.md
>
> [ ] +1 Release this as Apache DataFusion Comet 0.1.0
> [ ] +0
> [ ] -1 Do not release this as Apache DataFusion Comet 0.1.0 because...
>
> Here is my vote:
>
> +1
>
> [1]:
> https://github.com/apache/datafusion-comet/tree/df980bf67e59f439521b90dd2139c6a8d706c0bc
> [2]:
> https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-comet-0.1.0-rc2
> [3]:
> https://github.com/apache/datafusion-comet/blob/df980bf67e59f439521b90dd2139c6a8d706c0bc/CHANGELOG.md
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@datafusion.apache.org
> For additional commands, e-mail: dev-h...@datafusion.apache.org
>
>

Re: [VOTE] Release Apache DataFusion Comet 0.1.0 RC2

[Parth Chandra]

+1  (non-binding)
(resolved my previous comment). There is a `release-nogit` target for make.


On Mon, Jul 22, 2024 at 10:21 AM Parth Chandra  wrote:

> Src tarball verified successfully.
>
> However, building the artifacts from src gives an error because the
> makefile does not have "-Dmaven.gitcommitid.skip=true" specified.
> $ make release ( or just make)
> ...
> [ERROR] Failed to execute goal
> io.github.git-commit-id:git-commit-id-maven-plugin:4.9.9:revision
> (get-the-git-infos) on project comet-common-spark3.4_2.12: .git directory
> is not found! Please specify a valid [dotGitDirectory] in your pom.xml ->
> [Help 1]
>
>
> On Sat, Jul 20, 2024 at 1:56 PM Andy Grove 
> wrote:
>
>> Hi,
>>
>> I would like to propose a release of Apache DataFusion Comet version
>> 0.1.0.
>>
>> This release candidate is based on commit:
>> df980bf67e59f439521b90dd2139c6a8d706c0bc [1]
>> The proposed release tarball and signatures are hosted at [2].
>> The changelog is located at [3].
>>
>> Please download, verify checksums and signatures, run the unit tests, and
>> vote
>> on the release. The vote will be open for at least 72 hours.
>>
>> Only votes from PMC members are binding, but all members of the community
>> are
>> encouraged to test the release and vote with "(non-binding)".
>>
>> The standard verification procedure is documented at
>> https://github.com/apache/datafusion-comet/blob/main/dev/release/verifying-release-candidates.md
>>
>> [ ] +1 Release this as Apache DataFusion Comet 0.1.0
>> [ ] +0
>> [ ] -1 Do not release this as Apache DataFusion Comet 0.1.0 because...
>>
>> Here is my vote:
>>
>> +1
>>
>> [1]:
>> https://github.com/apache/datafusion-comet/tree/df980bf67e59f439521b90dd2139c6a8d706c0bc
>> [2]:
>> https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-comet-0.1.0-rc2
>> [3]:
>> https://github.com/apache/datafusion-comet/blob/df980bf67e59f439521b90dd2139c6a8d706c0bc/CHANGELOG.md
>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@datafusion.apache.org
>> For additional commands, e-mail: dev-h...@datafusion.apache.org
>>
>>

Re: [VOTE] Release Apache DataFusion Comet 0.3.0 RC1

[Parth Chandra]

+1 [non-binding]

Tested local build with artifacts published to maven.

On Tue, Sep 24, 2024 at 6:16 PM Andy Grove 
wrote:

> Hi,
>
> I would like to propose a release of Apache DataFusion Comet version 0.3.0.
>
> ** NOTE: we now have JAR files available for download for testing the
> release candidate. See notes below. **
>
> This release candidate is based on commit:
> 3783faaa01078a35bee93b299368f8c72869198d [1]
> The proposed release tarball and signatures are hosted at [2].
> The changelog is located at [3].
> Maven artifacts are available at [4].
>
> Please download, verify checksums and signatures, run the unit tests, and
> vote
> on the release. The vote will be open for at least 72 hours.
>
> Only votes from PMC members are binding, but all members of the community
> are
> encouraged to test the release and vote with "(non-binding)".
>
> The standard verification procedure is documented at
> https://github.com/apache/datafusion-comet/blob/main/dev/release/verifying-release-candidates.md
>
> [ ] +1 Release this as Apache DataFusion Comet 0.3.0
> [ ] +0
> [ ] -1 Do not release this as Apache DataFusion Comet 0.3.0 because...
>
> Here is my vote:
>
> +1
>
> [1]:
> https://github.com/apache/datafusion-comet/tree/3783faaa01078a35bee93b299368f8c72869198d
> [2]:
> https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-comet-0.3.0-rc1
> [3]:
> https://github.com/apache/datafusion-comet/blob/3783faaa01078a35bee93b299368f8c72869198d/CHANGELOG.md
> [4]:
> https://repository.apache.org/content/repositories/orgapachedatafusion-1010
>
>
>

Re: [DISCUSS] Enable GitHub Discussions for DataFusion Comet

[Parth Chandra]

+1 on github discussions.

Keeping in mind the following adage, I would also suggest that *decisions*
should be reposted on the mailing list -
" If it didn’t happen on a mailing list, it didn’t happen."  [1]

[1] https://incubator.apache.org/guides/committer.html#mailing_lists


On Mon, Sep 23, 2024 at 6:50 AM Andy Grove  wrote:

> I only noticed a question about Comet by chance yesterday because it was
> posted as a discussion in the core DataFusion repository.
>
> Perhaps we should enable discussions in the Comet repository as well.
>
> Please let me know your thoughts.
>
> Thanks,
>
> Andy.
>

Re: [VOTE] Release Apache DataFusion Comet 0.4.0 RC1

[Parth Chandra]

+1 (non binding)

Verified om M3 mac.


On Fri, Nov 15, 2024 at 5:31 PM L. C. Hsieh  wrote:

> +1 (binding)
>
> Verified on M3 Mac.
>
> Thanks Andy.
>
> On Fri, Nov 15, 2024 at 1:18 PM Andy Grove 
> wrote:
> >
> > Hi,
> >
> > I would like to propose a release of Apache DataFusion Comet version
> 0.4.0.
> >
> > This release candidate is based on commit:
> b74bfe4adc08c806b059cd8b85947b1fc15d7b9b [1]
> > The proposed release tarball and signatures are hosted at [2].
> > Pre-built jar files are available in a Maven staging repository [3].
> > The changelog is located at [4].
> >
> > Please download, verify checksums and signatures, run the unit tests,
> and vote
> > on the release. The vote will be open for at least 72 hours.
> >
> > Only votes from PMC members are binding, but all members of the
> community are
> > encouraged to test the release and vote with "(non-binding)".
> >
> > The standard verification procedure is documented at
> https://github.com/apache/datafusion-comet/blob/main/dev/release/verifying-release-candidates.md
> >
> > [ ] +1 Release this as Apache DataFusion Comet 0.4.0
> > [ ] +0
> > [ ] -1 Do not release this as Apache DataFusion Comet 0.4.0 because...
> >
> > Here is my vote:
> >
> > +1
> >
> > [1]:
> https://github.com/apache/datafusion-comet/tree/b74bfe4adc08c806b059cd8b85947b1fc15d7b9b
> > [2]:
> https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-comet-0.4.0-rc1
> > [3]:
> https://repository.apache.org/#nexus-search;quick~org.apache.datafusion
> > [4]:
> https://github.com/apache/datafusion-comet/blob/b74bfe4adc08c806b059cd8b85947b1fc15d7b9b/CHANGELOG.md
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@datafusion.apache.org
> > For additional commands, e-mail: dev-h...@datafusion.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@datafusion.apache.org
> For additional commands, e-mail: dev-h...@datafusion.apache.org
>
>

Re: [VOTE] Release Apache DataFusion Comet 0.5.0 RC1

[Parth Chandra]

+1 (non binding)
Verified release (M3 mac)
Verified artifacts in staging repo are accessible by maven (and Intellij)

Thank you Andy!



On Tue, Jan 14, 2025 at 4:39 PM kazuyuki tanimura
 wrote:

> +1 (non-binding)
>
> verified on M4
>
> Kazu
>
>
> > On Jan 14, 2025, at 12:47 PM, Andrew Lamb  wrote:
> >
> > +1 (binding)
> >
> > I verified on an m3 mac. Looks like another good release
> >
> > Thank you Andy
> >
> > On Tue, Jan 14, 2025 at 3:12 PM L. C. Hsieh  wrote:
> >
> >> +1 (binding)
> >>
> >> Verified on M3 Mac.
> >>
> >> Thanks Andy.
> >>
> >> On Tue, Jan 14, 2025 at 11:24 AM Andy Grove
> >>  wrote:
> >>>
> >>> Hi,
> >>>
> >>> I would like to propose a release of Apache DataFusion Comet version
> >> 0.5.0.
> >>>
> >>> This release candidate is based on commit:
> >> 698c6e5ee70404ea51151c3b4c927ea52d5c0bc2 [1]
> >>> The proposed release tarball and signatures are hosted at [2].
> >>> Pre-built jar files are available in a Maven staging repository [3].
> >>> The changelog is located at [4].
> >>>
> >>> Please download, verify checksums and signatures, run the unit tests,
> >> and vote
> >>> on the release. The vote will be open for at least 72 hours.
> >>>
> >>> Only votes from PMC members are binding, but all members of the
> >> community are
> >>> encouraged to test the release and vote with "(non-binding)".
> >>>
> >>> The standard verification procedure is documented at
> >>
> https://github.com/apache/datafusion-comet/blob/main/dev/release/verifying-release-candidates.md
> >>>
> >>> [ ] +1 Release this as Apache DataFusion Comet 0.5.0
> >>> [ ] +0
> >>> [ ] -1 Do not release this as Apache DataFusion Comet 0.5.0 because...
> >>>
> >>> Here is my vote:
> >>>
> >>> +1
> >>>
> >>> [1]:
> >>
> https://github.com/apache/datafusion-comet/tree/698c6e5ee70404ea51151c3b4c927ea52d5c0bc2
> >>> [2]:
> >>
> https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-comet-0.5.0-rc1
> >>> [3]:
> >> https://repository.apache.org/#nexus-search;quick~org.apache.datafusion
> >>> [4]:
> >>
> https://github.com/apache/datafusion-comet/blob/698c6e5ee70404ea51151c3b4c927ea52d5c0bc2/CHANGELOG.md
> >>>
> >>>
> >>> -
> >>> To unsubscribe, e-mail: dev-unsubscr...@datafusion.apache.org
> >>> For additional commands, e-mail: dev-h...@datafusion.apache.org
> >>>
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@datafusion.apache.org
> >> For additional commands, e-mail: dev-h...@datafusion.apache.org
> >>
> >>
>
>

Re: [VOTE] Release Apache DataFusion Comet 0.7.0 RC1

[Parth Chandra]

+1 (non-binding)
Verified on M3 mac.


On Thu, Mar 13, 2025 at 5:57 PM Andy Grove  wrote:

> Hi,
>
>
> I would like to propose a release of Apache DataFusion Comet version 0.7.0.
>
>
> This release candidate is based on commit:
> 664e681e425648699daeb615a4f8ffe2c91ae9ad [1]
>
> The proposed release tarball and signatures are hosted at [2].
>
> Pre-built jar files are available in a Maven staging repository [3].
>
> The changelog is located at [4].
>
>
> Please download, verify checksums and signatures, run the unit tests, and
> vote
>
> on the release. The vote will be open for at least 72 hours.
>
>
> Only votes from PMC members are binding, but all members of the community
> are
>
> encouraged to test the release and vote with "(non-binding)".
>
>
> The standard verification procedure is documented at
>
> https://github.com/apache/datafusion-comet/blob/main/dev/release/verifying-release-candidates.md
>
>
> [ ] +1 Release this as Apache DataFusion Comet 0.7.0
>
> [ ] +0
>
> [ ] -1 Do not release this as Apache DataFusion Comet 0.7.0 because...
>
>
> Here is my vote:
>
>
> +1
>
>
> [1]:
>
> https://github.com/apache/datafusion-comet/tree/664e681e425648699daeb615a4f8ffe2c91ae9ad
>
> [2]:
>
> https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-comet-0.7.0-rc1
>
> [3]:
> https://repository.apache.org/#nexus-search;quick~org.apache.datafusion
>
> [4]:
>
> https://github.com/apache/datafusion-comet/blob/664e681e425648699daeb615a4f8ffe2c91ae9ad/CHANGELOG.md
>

Re: [VOTE] Release Apache DataFusion Comet 0.8.0 RC1

[Parth Chandra]

+1.
Verified on Mac. Also checked maven artifacts with a small test app.

On Fri, Apr 25, 2025 at 8:05 AM Oleks V.  wrote:

> +1
>
> My bad forgot to vote
>
> On 2025/04/25 14:54:55 "Oleks V." wrote:
> > Verified on M3 Mac
> >
> > Thanks Andy
> >
> > On 2025/04/23 13:45:54 Andy Grove wrote:
> > > Hi,
> > >
> > >
> > > I would like to propose a release of Apache DataFusion Comet version
> 0.8.0.
> > >
> > >
> > > This release candidate is based on commit:
> > > 64b625233b8e0617687ea5b097344a805066861a [1]
> > >
> > > The proposed release tarball and signatures are hosted at [2].
> > >
> > > Pre-built jar files are available in a Maven staging repository [3].
> > >
> > > The changelog is located at [4].
> > >
> > >
> > > Please download, verify checksums and signatures, run the unit tests,
> and
> > > vote
> > >
> > > on the release. The vote will be open for at least 72 hours.
> > >
> > >
> > > Only votes from PMC members are binding, but all members of the
> community
> > > are
> > >
> > > encouraged to test the release and vote with "(non-binding)".
> > >
> > >
> > > The standard verification procedure is documented at
> > >
> https://github.com/apache/datafusion-comet/blob/main/dev/release/verifying-release-candidates.md
> > >
> > >
> > > [ ] +1 Release this as Apache DataFusion Comet 0.8.0
> > >
> > > [ ] +0
> > >
> > > [ ] -1 Do not release this as Apache DataFusion Comet 0.8.0 because...
> > >
> > >
> > > Here is my vote:
> > >
> > >
> > > +1
> > >
> > >
> > > [1]:
> > >
> https://github.com/apache/datafusion-comet/tree/64b625233b8e0617687ea5b097344a805066861a
> > >
> > > [2]:
> > >
> https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-comet-0.8.0-rc1
> > >
> > > [3]:
> https://repository.apache.org/#nexus-search;quick~org.apache.datafusion
> > >
> > > [4]: https://github.com/apache/datafusion-comet/pull/1675
> > >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@datafusion.apache.org
> > For additional commands, e-mail: dev-h...@datafusion.apache.org
> >
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@datafusion.apache.org
> For additional commands, e-mail: dev-h...@datafusion.apache.org
>
>