[DISCUSS] Adding a pull-style iterator API to the C data interface

2020-06-28 Thread Wes McKinney
In ARROW-8301 [1] and elsewhere we've been discussing how to communicate what amounts to a sequence of arrays or a sequence of RecordBatch objects using the C data interface. Example use cases: * Returning a sequence of record / row batches from a database driver * Sending a C++ arrow::ChunkedArr

Re: [DISCUSS] Removing top-level validity bitmap from Union type

2020-06-28 Thread Wes McKinney
I'm going to start a vote about this tomorrow if there are not more comments about this. On Thu, Jun 25, 2020 at 5:41 PM Wes McKinney wrote: > > I updated the PR to fix some issues with my edits that Antoine pointed > out. I can start working on a C++ patch to implement the C++ changes > in the n

Re: [VOTE] Add a "Feature" enum to Schema.fbs

2020-06-28 Thread Sutou Kouhei
+1 (binding) In "[VOTE] Add a "Feature" enum to Schema.fbs" on Sat, 27 Jun 2020 20:46:55 -0700, Micah Kornfield wrote: > Hi, > > As discussed on the mailing list [1] I would like to add a feature enum to > enhance our ability to evolve Arrow in a forward compatible manner and > allow clie

Re: [VOTE] Add a "Feature" enum to Schema.fbs

2020-06-28 Thread Neal Richardson
+1 (binding) Neal On Sun, Jun 28, 2020 at 7:35 AM Wes McKinney wrote: > +1 (binding) > > On Sat, Jun 27, 2020 at 10:47 PM Micah Kornfield > wrote: > > > > Hi, > > > > As discussed on the mailing list [1] I would like to add a feature enum > to > > enhance our ability to evolve Arrow in a forwa

Re: [Format][C++] Offering limited support for unsigned dictionary indices

2020-06-28 Thread Wes McKinney
I opened a PR https://github.com/apache/arrow/pull/7567 We should probably vote on this so I will wait another day or so before opening a vote so we can get this change through for the release On Sun, Jun 28, 2020 at 9:38 AM Wes McKinney wrote: > > I'll propose a patch to Columnar.rst about thi

Re: [DISCUSS] Incrementing Arrow MetadataVersion from V4 to V5 for 1.0.0 release

2020-06-28 Thread Wes McKinney
I opened a PR https://github.com/apache/arrow/pull/7566 We should prioritize getting through the other format changes, but we can vote on this in the meantime if there is consensus On Fri, Jun 26, 2020 at 2:58 PM Micah Kornfield wrote: > > I agree I think we have to do this given the number of c

Re: [Format][C++] Offering limited support for unsigned dictionary indices

2020-06-28 Thread Wes McKinney
I'll propose a patch to Columnar.rst about this. I think it's OK to allow uint64 indices (so we don't have to revisit the issue in a year or two or...) but the specification should say that they are not preferred. On Fri, Jun 26, 2020 at 7:20 PM Paul Taylor wrote: > > Responding to this comment f

Re: [VOTE] Add a "Feature" enum to Schema.fbs

2020-06-28 Thread Wes McKinney
+1 (binding) On Sat, Jun 27, 2020 at 10:47 PM Micah Kornfield wrote: > > Hi, > > As discussed on the mailing list [1] I would like to add a feature enum to > enhance our ability to evolve Arrow in a forward compatible manner and > allow clients and servers to negotiate which features are supporte

[NIGHTLY] Arrow Build Report for Job nightly-2020-06-28-0

2020-06-28 Thread Crossbow
Arrow Build Report for Job nightly-2020-06-28-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-06-28-0 Failed Tasks: - debian-stretch-arm64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-06-28-0-travis-debian-stretch-arm64 - tes