Re: [DISCUSS] Removing the "page" field from the Buffer record batch Arrow metadata

2017-10-21 Thread Phillip Cloud
+1 agree it should be solved separately. On Fri, Oct 20, 2017, 21:24 Wes McKinney wrote: > Patch here: https://github.com/apache/arrow/pull/1225 > > On Fri, Oct 20, 2017 at 12:13 AM, Li Jin wrote: > > +1 for the change too. >

Re: [DISCUSS] Removing the "page" field from the Buffer record batch Arrow metadata

2017-10-20 Thread Wes McKinney
Patch here: https://github.com/apache/arrow/pull/1225 On Fri, Oct 20, 2017 at 12:13 AM, Li Jin wrote: > +1 for the change too.

Re: [DISCUSS] Removing the "page" field from the Buffer record batch Arrow metadata

2017-10-19 Thread Li Jin
+1 for the change too.

Re: [DISCUSS] Removing the "page" field from the Buffer record batch Arrow metadata

2017-10-19 Thread Philipp Moritz
+1 for the change I'm all for making the metadata small and to solve it in a different if the field is really needed. Users who do not need the feature shouldn't have to pay for it. On Thu, Oct 19, 2017 at 6:02 PM, Wes McKinney wrote: > The JIRA for this is https://issues.apache.org/jira/browse

Re: [DISCUSS] Removing the "page" field from the Buffer record batch Arrow metadata

2017-10-19 Thread Wes McKinney
The JIRA for this is https://issues.apache.org/jira/browse/ARROW-1409. I will wait a little while for others to weigh in, but after that I can write a patch to remove the attribute and bump the metadata format version number. On Thu, Oct 19, 2017 at 4:37 PM, Bryan Cutler wrote: > +1, sounds ok to

Re: [DISCUSS] Removing the "page" field from the Buffer record batch Arrow metadata

2017-10-19 Thread Bryan Cutler
+1, sounds ok to me to try to solve this problem a different way in the future once needed. On Thu, Oct 19, 2017 at 12:30 PM, Jacques Nadeau wrote: > Seems reasonable. I was among those that originally argued for this field > but given that we haven't used it yet, I think your proposal makes sen

Re: [DISCUSS] Removing the "page" field from the Buffer record batch Arrow metadata

2017-10-19 Thread Jacques Nadeau
Seems reasonable. I was among those that originally argued for this field but given that we haven't used it yet, I think your proposal makes sense. +1 On Wed, Oct 18, 2017 at 5:40 PM, Wes McKinney wrote: > When we originally drafted the metadata for record batches, we > included a "page id" in