Re: [VOTE][Format] Add experimental ArrowDeviceArray to C-Data API

@Antoine: I've updated the PR with a prose description of the C Device Data interface. Sorry for the lack of that in the first place. --Matt On Tue, May 23, 2023 at 10:34 AM Antoine Pitrou wrote: > > Also, I forgot to say, but thanks a lot for doing this! We can hope this > will drastically imp

Re: [ANNOUNCE] New Arrow committer: Gang Wu

Congrats Gang! On Mon, May 15, 2023, 9:57 AM Ian Cook wrote: > Congratulations Gang! > > On Mon, May 15, 2023 at 9:47 AM vin jake wrote: > > > > Congrats Gang! > > > > On Mon, May 15, 2023 at 9:33 PM Sutou Kouhei wrote: > > > > > On behalf of the Arrow PMC, I'm happy to announce that Gang > >

Re: [VOTE][Format][Flight] Ordered data support

+1 Thanks Kou! On Thu, May 4, 2023 at 6:00 AM Andrew Lamb wrote: > +1 > > Thanks Kou! > > On Thu, May 4, 2023 at 6:57 AM David Li wrote: > > > +1 > > > > Thanks Kou! > > > > On Thu, May 4, 2023, at 05:09, Jean-Baptiste Onofré wrote: > > > +1 (non binding) > > > > > > Regards > > > JB > > > > >

Re: [VOTE] Release Apache Arrow 12.0.0 - RC0

/etc/localtime symbolic link wasn't being set in the container and is leveraged by the Orc adapter tests. So setting the localtime (to any valid zone info) was sufficient to let the tests run and pass. Hope this helps! On Fri, Apr 28, 2023 at 11:49 AM Matthew Topol wrote: > Looks like th

Re: [VOTE] Release Apache Arrow 12.0.0 - RC0

read. I'll report back if it solves the issue. --Matt On Fri, Apr 28, 2023 at 11:29 AM Matthew Topol wrote: > @Kou: I was able to reproduce the libcrypto failure that Jacob saw using > https://hub.docker.com/r/manjarolinux/base though i did need to manually > install git first since

Re: [VOTE] Release Apache Arrow 12.0.0 - RC0

@Kou: I was able to reproduce the libcrypto failure that Jacob saw using https://hub.docker.com/r/manjarolinux/base though i did need to manually install git first since it doesn't come with it. $ pacman -Syu git $ git clone https://github.com/apache/arrow.git $ cd arrow $ TEST_DEFAULT=0 TEST_SOUR

Re: [VOTE] Formalize how to change format

+1 (Non-binding) On Mon, Apr 24, 2023 at 9:12 AM Neal Richardson wrote: > +1, thanks for taking the initiative on this! > > Neal > > On Mon, Apr 24, 2023 at 6:10 AM Andrew Lamb wrote: > > > +1 > > > > > > On Mon, Apr 24, 2023 at 1:55 AM David Li wrote: > > > > > +1 > > > > > > Thanks Kou! > >

Re: [ANNOUNCE] New Arrow committer: Mustafa Akur

Congrats Mustafa! Welcome! On Fri, Mar 31, 2023 at 9:24 AM David Li wrote: > Congrats & welcome Mustafa! > > On Fri, Mar 31, 2023, at 06:21, Andrew Lamb wrote: > > Hello Arrow Community > > > > On behalf of the Arrow PMC, I'm happy to announce that Mustafa Akur > > has accepted an invitation to

Re: [DISCUSS] Flight RPC/Flight SQL/ADBC enhancements

ues could > indicate support for each of them.) > > On Thu, Feb 16, 2023, at 16:16, Matthew Topol wrote: > > While implementing Transaction handling for ADBC via Flight SQL's > > transaction primitives, another potential enhancement would be to expand > > the

Re: [DISCUSS] Flight RPC/Flight SQL/ADBC enhancements

While implementing Transaction handling for ADBC via Flight SQL's transaction primitives, another potential enhancement would be to expand the BeginTransaction request to include a spot for "options" such as IsolationLevel or marking a transaction as ReadOnly. Anyone have thoughts on this? On Wed

Re: [VOTE] Release Apache Arrow ADBC 0.2.0 - RC1

So I just ran it again this evening, without actually changing anything that I'm aware of, and everything worked perfectly fine without any issues... *shrug* I'm gonna take the win and vote +1 (non-binding) :) huzzah! On Sat, Feb 11, 2023 at 4:37 PM Sutou Kouhei wrote: > Hi, > > Sorry. I misse

Re: [DISCUSS] The default commit message for merge button

(non-binding) I am also +1 for title + description On Tue, Jan 31, 2023 at 11:17 AM Neal Richardson < neal.p.richard...@gmail.com> wrote: > +1, with or without description > > Neal > > On Tue, Jan 31, 2023 at 11:04 AM Weston Pace > wrote: > > > +1 to both. This only applies to the merge bu

Re: [Monorepo] Add labels breaking-change and critical-fix

> To answer Matt's comment, though: those are not necessarily the criteria for minor releases, if we think of the latter as bugfix releases. When we do bugfixes, we release them as Patch releases (which I'd argue is correct). In an ideal world, we'd only do a *major* version release when there are

Re: Apache Arrow Board Report, by Jan 11 2023

I added some comments and updates for the Go module. I didn't know how specific or detailed to get, so let me know if I should elaborate more on it or something. --Matt On Tue, Dec 27, 2022 at 4:45 PM Andrew Lamb wrote: > Hello Arrow Community, > > One of the (possibly the only) responsibilitie

Re: [VOTE] Add RLE Arrays to Arrow Format

On Mon, Dec 19, 2022 at 3:32 PM Matthew Topol > wrote: > > > > Okay, slight edit to my previous email: It was brought to my attention > that > > we need at least 3 +1 binding votes, so this vote is still open for the > > moment. > > > > @IanCook: With the ch

Re: [VOTE] Add RLE Arrays to Arrow Format

Okay, slight edit to my previous email: It was brought to my attention that we need at least 3 +1 binding votes, so this vote is still open for the moment. @IanCook: With the change of the name to RunEndEncoding is that sufficient to change your vote to a +1? On Mon, Dec 19, 2022 at 12:57 PM Matt

Re: Request for Patch release of 10.0.2

, Dec 12, 2022 at 10:09 AM Matthew Topol wrote: > I'm in favor of this, can someone from the PMC please comment to the > viability of this? I know we have the v11 release coming up in the next > month or so and we'd need someone to be the release manager for performing > a

Re: Request for Patch release of 10.0.2

I'm in favor of this, can someone from the PMC please comment to the viability of this? I know we have the v11 release coming up in the next month or so and we'd need someone to be the release manager for performing a v10.0.2 patch release. So I'm not sure whether or not this is a viable option. -

Re: [VOTE] Disable ASF Jira issue reporting

+1 On Fri, Nov 25, 2022 at 10:31 AM Alessandro Molina wrote: > +1 as far as for "now" we actually mean "as soon as the necessary scripts > have been ported to github" > > I mean, I doubt the plan is to disable jira before we can actually ship PRs > from github issues and thus block development.

Re: [ANNOUNCE] New Arrow PMC member: Raphael Taylor-Davies

Congrats!! On Tue, Sep 20, 2022 at 7:23 PM Wes McKinney wrote: > Congratulations! > > On Tue, Sep 20, 2022 at 12:37 PM Ashish wrote: > > > > Congratulations !! > > > > On Tue, Sep 20, 2022 at 10:17 AM Ian Joiner > wrote: > > > > > Congrats Raphael! > > > > > > On Mon, Sep 19, 2022 at 9:56 PM S

Re: [VOTE] Adopt ADBC database client connectivity specification

+1 (non-binding)! On Wed, Sep 21, 2022 at 1:34 PM Dewey Dunnington wrote: > +1! (non-binding) > > On Wed, Sep 21, 2022 at 12:47 PM Gavin Ray wrote: > > > +1 (non-binding/I'm not important) > > > > On Wed, Sep 21, 2022 at 11:40 AM David Li wrote: > > > > > Hello, > > > > > > We have been discus

Re: PRs for RLE support

ary member could be the values array and perhaps make it easier for implementations that already handle dictionaries. On Wed, Sep 14, 2022 at 2:04 PM Matthew Topol mailto:m...@voltrondata.com.invalid>> wrote: > Just wanted to chime in here that I also have several draft PRs for

Re: PRs for RLE support

Just wanted to chime in here that I also have several draft PRs for implementing the RLE arrays in Go as the second implementation (since we use two implementations as a requirement to vote on changes/additions to the format). They can be found here:

Re: [DISC] Improving Arrow's database support

tps://github.com/apache/arrow-adbc/issues/55> [3]: <https://github.com/apache/arrow-adbc/issues/59> On Sat, Sep 3, 2022, at 18:36, Matthew Topol wrote: > +1 from me on the strategy proposed by Kou. > > That would be my preference also. I agree it is preferable to be versioned >

Re: [VOTE] Substrait for Flight SQL

I'm not PMC but i'll give a +1 (non-binding) vote. I like the idea of integrating Substrait plans into Flight SQL if possible and it aligns with the arrow-adbc work. On Thu, Sep 8 2022 at 11:31:59 AM -0400, David Li wrote: My vote: +1 (binding) Are any other PMC members available to take a

Re: [DISC] Improving Arrow's database support

+1 from me on the strategy proposed by Kou. That would be my preference also. I agree it is preferable to be versioned independently. --Matt On Sat, Sep 3, 2022, 6:24 PM Sutou Kouhei wrote: > Hi, > > > Do we have a preference for versioning strategy? Should we > > proceed in lockstep with the

Re: Usage of the name Feather?

When I wrote "In-Memory Analytics with Apache Arrow" I definitely treated "Feather" as deprecated and mentioned it only in passing specifically indicating "Arrow IPC" as the terminology to use. I only even mentioned "Feather" at all because there are still methods in pyarrow that reference it b

Re: DISCUSS: [C++] Switch to C++17

I can definitely vouch for the benefits of upgrading to C++17, it makes general development significantly more enjoyable! On that end, I second Neal's suggestion for deployment, that's roughly the same plan we used in my past experience when performing a compiler upgrade on a very large codeba

Re: [RESULT][VOTE] Release Apache Arrow 8.0.1 - RC0

It would probably be a good idea to just make sure that any release notes mention that for a Go user to upgrade their dependency they need to run something like `go get -u github.com/apache/arrow/go/v6/@v6.0.2` replacing v6/v6.0.2 with their desired version combination. This will get them the p

Re: Arrow Flight usage with graph databases

JSON structured records). > Those Python record objects are then converted into an Arrow Formatted > Table using pa.Table.from_pylist(). > The Arrow Table is then sent back to the client to complete the original > Fight RPC request. > > -Original Message- > From: Matthew Top

Re: Arrow Flight usage with graph databases

So this is sightly different than what I was doing and spoke about. As far as I can tell from your links, you are evaluating the graphql using that graphql server and then converting the JSON response into arrow format (correct me if I'm wrong please). What I did was to hook into a graphql parser

Re: Proposal: renaming the 'master' branch to 'main'

I'm in favor of it, for what it's worth. --Matt On Mon, Jul 25 2022 at 02:56:31 PM -0600, Wes McKinney wrote: hi all, Do you think we could make a push to make this happen after the 9.0.0 release goes out? Thanks Wes On Tue, Feb 15, 2022 at 2:32 PM Fiona La >

RE: [DISCUSS] A book about Apache Arrow

ut just curious whether there's a preview > available/table-of-contents though? I didn't see one on either Amazon > or Packt. > > Thanks =) > > On Wed, Apr 20, 2022 at 1:26 PM Matthew Topol wrote: > >> I was wondering who they got as the technical reviewe

RE: [DISCUSS] A book about Apache Arrow

: [DISCUSS] A book about Apache Arrow Hi Matt, FYI I'm currently a reviewer on this book already. On Wed, Apr 20, 2022 at 9:51 AM Matthew Topol wrote: > Hey All, > > I've been writing a book on Apache Arrow for Packt (packtpub.com) and > it's in the final stages of re

[DISCUSS] A book about Apache Arrow

rnes & Noble for Pre-order (https://www.amazon.com/Memory-Analytics-Apache-Arrow-hierarchical-dp-1801071039/dp/1801071039/). Thanks much for your time everyone! Hope to hear back from someone soon. Matthew Topol Vice President, Principal Software Architect mto...@factset.com<mailto:mto...@

RE: [VOTE][RESULT] Release Apache Arrow 6.0.0 - RC3

- RC3 On Tue, Nov 2, 2021 at 7:27 PM Matthew Topol wrote: > > The patch to the release-6.0.0 branch to fix the Go issue has been merged and > is ready for a 6.0.1 release once everything else is. Actually the release branches are only used for the actual release, this allows the mai

RE: [VOTE][RESULT] Release Apache Arrow 6.0.0 - RC3

The patch to the release-6.0.0 branch to fix the Go issue has been merged and is ready for a 6.0.1 release once everything else is. --Matt -Original Message- From: Ian Cook Sent: Friday, October 29, 2021 11:48 AM To: dev@arrow.apache.org Subject: Re: [VOTE][RESULT] Release Apache Arrow

[Discuss][Release][Go] Major Version handling for Go

It was suggested that I should advertise this in the dev mailing list as it is a significant change. When developing Go modules, if you do a major version release you need to update the module import path to add a `v#` suffix in order for the go tooling to handle the version properly. Up until

RE: [VOTE][RESULT] Release Apache Arrow 6.0.0 - RC3

There was also an issue found with the Go package deployment which prevents actually using `go get -u github.com/apache/arrow/go/arrow@v6.0.0` . The fix is in https://github.com/apache/arrow/pull/11566. I Second the request for a 6.0.1 release with a patch before the release post goes out to ens

RE: [DISCUSS] Use conbench.ursa.dev for arrow-rs and arrow-datafusion

This info is really useful for other languages that want to publish to conbench also, I'd love to find some time to try to also publish the Go benchmarks there too, luckily the go impl is already in the main arrow repo, so other than figuring out how to do the publishing, I'd just need Wes' bles

RE: [ANNOUNCE] New Arrow committer: Nic Crane

Congrats! -Original Message- From: Krisztián Szűcs Sent: Thursday, September 9, 2021 3:47 PM To: dev@arrow.apache.org Subject: Re: [ANNOUNCE] New Arrow committer: Nic Crane Congrats Nic! On 2021. Sep 9., Thu at 21:31, Daniël Heres wrote: > Congrats, Nic! > > On Thu, Sep 9, 2021, 21:2

RE: [ANNOUNCE] New Arrow committer: Matt Topol

Thanks so much everyone! --Matt -Original Message- From: Rok Mihevc Sent: Tuesday, August 31, 2021 7:54 AM To: dev@arrow.apache.org Subject: Re: [ANNOUNCE] New Arrow committer: Matt Topol Congrats! On Tue, Aug 31, 2021 at 1:22 PM David Li wrote: > > Congrats, Matt! > > -David > > On

RE: [C++][Go] CGO For Dataset API Integration

om: Antoine Pitrou > Sent: Monday, August 23, 2021 2:00 PM > To: dev@arrow.apache.org > Subject: Re: [C++][Go] CGO For Dataset API Integration > > > Le 23/08/2021 à 19:53, Matthew Topol a écrit : >> The only thing I don't like it being a private module in the Go >> impl

RE: [C++][Go] CGO For Dataset API Integration

e: [C++][Go] CGO For Dataset API Integration Le 23/08/2021 à 19:53, Matthew Topol a écrit : > The only thing I don't like it being a private module in the Go > implementation is distribution. For native go code, consumers can just > perform `go get` and have it work. But for this i

RE: [C++][Go] CGO For Dataset API Integration

1:24 PM To: dev@arrow.apache.org Subject: Re: [C++][Go] CGO For Dataset API Integration Le 23/08/2021 à 19:16, Matthew Topol a écrit : > Unfortunately, Go currently can only integrate with C++ libraries through a C > interface. There does exist SWIG which is a generator for creating i

RE: [C++][Go] CGO For Dataset API Integration

ementation, only for using the Dataset API stuff (and potentially the compute library). --Matt -Original Message- From: Antoine Pitrou Sent: Monday, August 23, 2021 12:56 PM To: dev@arrow.apache.org Subject: Re: [C++][Go] CGO For Dataset API Integration Le 23/08/2021 à 18:22, Matt

RE: [C++][Go] CGO For Dataset API Integration

wbXLzU2HxxKFtP-yDuTQLu1SV0sjKs$ The first thing to do would be for the Go implementation to implement the C data interface. Regards Antoine. Le 23/08/2021 à 16:07, Matthew Topol a écrit : > Hey All, > > So I've been working on a use case where I needed to be able to use the > D

[C++][Go] CGO For Dataset API Integration

Hey All, So I've been working on a use case where I needed to be able to use the Dataset API from Golang and instead of trying to port all of it to Golang (which would require porting the Compute side too) I decided to create a proof of concept using CGO to just call into the existing C++ code

RE: [Go] Flight client app metadata access

I have put up: https://github.com/apache/arrow/pull/10142 We can shift any further discussion to the Github PR / Jira card now. -Original Message- From: Matthew Topol Sent: Friday, April 23, 2021 10:07 AM To: 'dev@arrow.apache.org' Subject: RE: [Go] Flight client app metad

RE: [Go] Flight client app metadata access

with the workaround I'm using: https://urldefense.com/v3/__https://gist.github.com/pgwhalen/ed768e18917610b2de7942144068f205__;!!PBKjc0U4!a9YpgQraXdWigY05L6LnmQdDQVLO6_pAavzW-0lWtm6rqtgEcZv82o0kRChMDEU$ (not particularly well tested and a little sloppy design, but seems to work). Paul On T

RE: [Go] Flight client app metadata access

You're absolutely correct and not missing anything, this is definitely an opportunity to make the flight record reader a bit more useful. I like the idea of the getLatestMetadata that you mentioned is on the java side. Can you file a JIRA issue for this? Given that I'm doing a lot of updates to

RE: [Go] expose ability to write arrow.Table to JSON

Micah is correct, the arrjson package is used for the internal integration testing using the specific JSON format for that integration testing which is not likely what Users would want when converting Arrow to JSON. There is not currently a recommended way to serialize an instance of arrow.Tabl

RE: [VOTE] Release Apache Arrow 4.0.0 - RC0

8, 2021 at 6:52 PM Matthew Topol wrote: > > Hey All, > > So I'm not familiar enough with the release scripts to do this myself and > offer up a PR, but could the release config be modified to also create the > appropriate tags for Go module versioning with the releases?

RE: [VOTE] Release Apache Arrow 4.0.0 - RC0

Hey All, So I'm not familiar enough with the release scripts to do this myself and offer up a PR, but could the release config be modified to also create the appropriate tags for Go module versioning with the releases? In the case of the Arrow Repo these would be "go/arrow/v4.0.0-rc0" and "go/p