Re: [DISCUSS][C++] Enabling finer-grained parallelism in compute operators, quantifying ExecBatch overhead

2021-08-18 Thread Wes McKinney
Now that we have the benchmark, it seems like it would be a good idea to try to devise possible solutions to this issue. I recognize that the particular interface of ExecBatchIterator may not be something we want to preserve, but we could instead focus on the general batch-splitting problem for pur

[R] how to become a contributor

2021-08-18 Thread Calvin Chiang
sorry as this is probably obvious to everyone but me. I'd like to become a contributor to the Apache Airflow R section. some quick qs: - How do i join? - are there any good descriptions of setting up a development environment for use across C++ and R? - is there any good material for

Re: [VOTE][Format] Add in a new interval type can combines Month, Days and Nanoseconds

2021-08-18 Thread Antoine Pitrou
+1 (binding) Le 17/08/2021 à 21:49, Micah Kornfield a écrit : Hello, As discussed previously [1], I'd like to call a vote to add a new interval type which is a triple of Month, Days, and nanoseconds. The formal definition is defined in a PR [2] along with Java and C++ implementations that have

Re: [R] how to become a contributor

2021-08-18 Thread Neal Richardson
On Wed, Aug 18, 2021 at 5:31 AM Calvin Chiang wrote: > sorry as this is probably obvious to everyone but me. > I'd like to become a contributor to the Apache Airflow R section. > Just to be clear, you do mean Arrow and not Airflow, right? > some quick qs: > >- How do i join? > You're alre

Re: [VOTE][Format] Add in a new interval type can combines Month, Days and Nanoseconds

2021-08-18 Thread Neal Richardson
+1 On Wed, Aug 18, 2021 at 6:06 AM Antoine Pitrou wrote: > +1 (binding) > > > Le 17/08/2021 à 21:49, Micah Kornfield a écrit : > > Hello, > > As discussed previously [1], I'd like to call a vote to add a new > interval > > type which is a triple of Month, Days, and nanoseconds. The formal > > d

Arrow sync call August 18 at 12:00 US/Eastern, 16:00 UTC

2021-08-18 Thread Ian Cook
Hi all, Our biweekly sync call is today at 12:00 noon Eastern time. For today's call, let's please us this Google Meet URL (different from the usual one): https://meet.google.com/ebp-tczo-xjn All are welcome to join. Notes will be shared with the mailing list afterward. Thanks, Ian

Flight SQL

2021-08-18 Thread Kyle Porter
Hello All, We've been working on adding Flight SQL as a formal part of Arrow Flight and the PR is available for review at https://github.com/apache/arrow/pull/10906. Most of the spec is now in place and, if you are interested, it would be good to have more eyes on the PR. Thanks! *Kyle Porter*

Re: [DISCUSS] Binary Values in Key value pairs WAS: Re: [INFO_REQUEST][FLIGHT] - Dynamic schema changes in ArrowFlight streams

2021-08-18 Thread David Li
This isn't too thought out yet but: 1. Any file which stuffs binary data into the value is already unreadable for anyone directly using Flatbuffers. So we can specify that the field must be valid UTF-8, but implementations can permit relaxed validation/reading as binary data instead in order t

Re: [VOTE][Format] Add in a new interval type can combines Month, Days and Nanoseconds

2021-08-18 Thread David Li
+1 On Wed, Aug 18, 2021, at 10:31, Neal Richardson wrote: > +1 > > On Wed, Aug 18, 2021 at 6:06 AM Antoine Pitrou wrote: > > > +1 (binding) > > > > > > Le 17/08/2021 à 21:49, Micah Kornfield a écrit : > > > Hello, > > > As discussed previously [1], I'd like to call a vote to add a new > > inter

Re: Arrow sync call August 18 at 12:00 US/Eastern, 16:00 UTC

2021-08-18 Thread Ian Cook
Meeting notes: Attendees: Ian Cook Nic Crane Alenka Frim David Li Rok Mihevc Niranda Perera Eduardo Ponce Discussion: - Voltron Data update - Ursa Computing now part of Voltron Data [1] - Ursa Labs now Voltron Labs - Mission to support Apache Arrow project and community continues - Sugges

[Website] [Rust] Blog post for DataFusion 5.0.0 and Ballista 0.5.0 releases

2021-08-18 Thread Andy Grove
The Rust community has been collaborating on two brief blog posts to announce the recent releases of DataFusion and Ballista. There is a PR up against the arrow-site repo [1]. I will merge this later this week if there are no objections. Thanks, Andy. [1] https://github.com/apache/arrow-site/p

Re: [R] how to become a contributor

2021-08-18 Thread Calvin Chiang
Hi Neal Ooops thanks! yes i meant Arrow :) Great thanks for the link to the dev environment setup - not sure how i missed that one. i'll check out some jiras and get started! On Wed, 18 Aug 2021 at 16:31, Neal Richardson wrote: > On Wed, Aug 18, 2021 at 5:31 AM Calvin Chiang > wrote: > > > s