Nice work!
On Fri, Sep 13, 2019 at 10:43 AM Wes McKinney wrote:
>
> Integration tests are passing:
>
> https://travis-ci.org/apache/arrow/jobs/584361357
>
> I'm merging the feature branch into master. Thanks all
>
> On Thu, Sep 12, 2019 at 6:15 PM Wes McKinney wrote:
> >
> > Thanks all!
> >
> >
Integration tests are passing:
https://travis-ci.org/apache/arrow/jobs/584361357
I'm merging the feature branch into master. Thanks all
On Thu, Sep 12, 2019 at 6:15 PM Wes McKinney wrote:
>
> Thanks all!
>
> It looks like all the changes are in, so we need to see if the
> integration tests pass
Thanks all!
It looks like all the changes are in, so we need to see if the
integration tests pass before we can merge these changes into master
On Thu, Sep 12, 2019 at 7:19 AM Sebastien Binet wrote:
>
> hi there,
>
> On Thu, Sep 12, 2019 at 12:45 AM Wes McKinney wrote:
>
> > Thanks Bryan.
> >
>
hi there,
On Thu, Sep 12, 2019 at 12:45 AM Wes McKinney wrote:
> Thanks Bryan.
>
> I merged the Java patch with the EOS change and submitted a C++ patch
> which also updates the specification
>
> https://github.com/apache/arrow/pull/5361
>
> Let me know when the JS or C# patches are ready to go
ber
> >> > the state, so if it reads the continuation token from the beginning,
> >> then
> >> > read all 8 bytes at the end.
> >> > >
> >> > > Thanks,
> >> > > Ji Liu
> >> > >
> >>
(星期四) 07:16
>> > > To:dev@arrow.apache.org ; Ji Liu <
>> > niki...@aliyun.com>
>> > > Cc:emkornfield ; Paul Taylor <
>> ptay...@apache.org>
>> > > Subject:RE: [RESULT] [VOTE] Alter Arrow binary protocol to address
>> > 8-byte Flatbuffer
beginning, then
> > read all 8 bytes at the end.
> > >
> > > Thanks,
> > > Ji Liu
> > >
> > > [1] https://github.com/apache/arrow/pull/5229
> > > [2] https://github.com/apache/arrow/pull/5229#discussion_r321715682
> > >
> > >
> > >
> > >
> > > --------------
# PR is up.
> >
> > https://github.com/apache/arrow/pull/5280
> >
> > Eric
> >
> > -----Original Message-
> > From: Eric Erhardt
> > Sent: Wednesday, September 4, 2019 10:12 AM
> > To: dev@arrow.apache.org; Ji Liu
> > Cc: emkornfield ; Paul
; Send Time:2019年9月5日(星期四) 07:16
> To:dev@arrow.apache.org ; Ji Liu
> Cc:emkornfield ; Paul Taylor
> Subject:RE: [RESULT] [VOTE] Alter Arrow binary protocol to address 8-byte
> Flatbuffer alignment requirements (2nd vote)
>
> The C# PR is up.
>
> https://github.com/apache/arro
--Original Message-
From: Wes McKinney
Sent: Tuesday, September 3, 2019 10:17 PM
To: Ji Liu
Cc: emkornfield ; dev ; Paul
Taylor
Subject: Re: [RESULT] [VOTE] Alter Arrow binary protocol to address 8-byte
Flatbuffer alignment requirements (2nd vote)
hi folks,
We now have patches up for Java,
-
From: Wes McKinney
Sent: Tuesday, September 3, 2019 10:17 PM
To: Ji Liu
Cc: emkornfield ; dev ; Paul
Taylor
Subject: Re: [RESULT] [VOTE] Alter Arrow binary protocol to address 8-byte
Flatbuffer alignment requirements (2nd vote)
hi folks,
We now have patches up for Java, JS, and Go.
eptember 3, 2019 10:17 PM
To: Ji Liu
Cc: emkornfield ; dev ; Paul
Taylor
Subject: Re: [RESULT] [VOTE] Alter Arrow binary protocol to address 8-byte
Flatbuffer alignment requirements (2nd vote)
hi folks,
We now have patches up for Java, JS, and Go. How are we doing on the code
reviews for gettin
> From:Ji Liu
> Send Time:2019年8月28日(星期三) 17:34
> To:emkornfield ; dev
> Cc:Paul Taylor
> Subject:Re: [RESULT] [VOTE] Alter Arrow binary protocol to address 8-byte
> Flatbuffer alignment requirements (2nd vote)
>
> I could take the Java implementation and wi
] [VOTE] Alter Arrow binary protocol to address 8-byte
Flatbuffer alignment requirements (2nd vote)
I could take the Java implementation and will take a close watch on this issue
in the next few days.
Thanks,
Ji Liu
--
From:Micah
Arrow binary protocol to address 8-byte
Flatbuffer alignment requirements (2nd vote)
I should have integration tests with 0.14.1 generated binaries in the next
few days. I think the one remaining unassigned piece of work in the Java
implementation, i can take that up next if no one else gets to
I should have integration tests with 0.14.1 generated binaries in the next
few days. I think the one remaining unassigned piece of work in the Java
implementation, i can take that up next if no one else gets to it.
On Tue, Aug 27, 2019 at 7:19 PM Wes McKinney wrote:
> Here's the C++ changes
>
>
Here's the C++ changes
https://github.com/apache/arrow/pull/5211
I'm going to create a integration branch where we can merge each patch
before merging to master
On Fri, Aug 23, 2019 at 9:03 AM Wes McKinney wrote:
>
> It isn't implemented in C++ yet but I will try to get a patch up for
> that so
It isn't implemented in C++ yet but I will try to get a patch up for
that soon (today maybe). I think we should create a branch where we
can stack the patches that implement this for each language.
On Fri, Aug 23, 2019 at 4:04 AM Paul Taylor wrote:
>
> I'll do the JS updates. Is it safe to valida
I'll do the JS updates. Is it safe to validate against the Arrow C++
integration tests?
On 8/22/19 7:28 PM, Micah Kornfield wrote:
I created https://issues.apache.org/jira/browse/ARROW-6313 as a tracking
issue with sub-issues on the development work. So far no-one has claimed
Java and Javascr
I created https://issues.apache.org/jira/browse/ARROW-6313 as a tracking
issue with sub-issues on the development work. So far no-one has claimed
Java and Javascript tasks.
Would it make sense to have a separate dev branch for this work?
Thanks,
Micah
On Thu, Aug 22, 2019 at 3:24 PM Wes McKinne
The vote carries with 4 binding +1 votes and 1 non-binding +1
I'll merge the specification patch later today and we can begin
working on implementations so we can get this done for 0.15.0
On Tue, Aug 20, 2019 at 12:30 PM Bryan Cutler wrote:
>
> +1 (non-binding)
>
> On Tue, Aug 20, 2019, 7:43 AM
+1 (non-binding)
On Tue, Aug 20, 2019, 7:43 AM Antoine Pitrou wrote:
>
> Sorry, had forgotten to send my vote on this.
>
> +1 from me.
>
> Regards
>
> Antoine.
>
>
> On Wed, 14 Aug 2019 17:42:33 -0500
> Wes McKinney wrote:
> > hi all,
> >
> > As we've been discussing [1], there is a need to int
Sorry, had forgotten to send my vote on this.
+1 from me.
Regards
Antoine.
On Wed, 14 Aug 2019 17:42:33 -0500
Wes McKinney wrote:
> hi all,
>
> As we've been discussing [1], there is a need to introduce 4 bytes of
> padding into the preamble of the "encapsulated IPC message" format to
> en
here is my vote: +1
On Tue, Aug 20, 2019 at 3:33 PM Wes McKinney wrote:
> We need some more PMC members to look at this vote. I know the issue
> is esoteric but please let us know if you have questions.
>
> On Thu, Aug 15, 2019 at 3:35 AM Micah Kornfield
> wrote:
> >
> > >
> > >
> > > Actually
We need some more PMC members to look at this vote. I know the issue
is esoteric but please let us know if you have questions.
On Thu, Aug 15, 2019 at 3:35 AM Micah Kornfield wrote:
>
> >
> >
> > Actually, it looks like my Mac's version of UBSAN doesn't detect the issue
> > at all. I will try on
>
>
> Actually, it looks like my Mac's version of UBSAN doesn't detect the issue
> at all. I will try on linux a by EOD.
Actually, the issue was I had alignment checks off. I verify this works
and it appears there is a UBSan issue with flatbuffers::Verify (I'll try to
see if I can find the issue
>
> I verified with these changes [1], without backwards compatibility
> support, UBSAN runs cleanly for IPC tests in C++
Actually, it looks like my Mac's version of UBSAN doesn't detect the issue
at all. I will try on linux a by EOD.
On Thu, Aug 15, 2019 at 12:52 AM Micah Kornfield
wrote:
> +
+1
I verified with these changes [1], without backwards compatibility support,
UBSAN runs cleanly for IPC tests in C++
Just wanted to clarify:
> Additionally with this vote, we want to formally approve the change to
> the Arrow "file" format to always write the (new 8-byte) end-of-stream
> marke
hi all,
As we've been discussing [1], there is a need to introduce 4 bytes of
padding into the preamble of the "encapsulated IPC message" format to
ensure that the Flatbuffers metadata payload begins on an 8-byte
aligned memory offset. The alternative to this would be for Arrow
implementations whe
I'm canceling the vote until we can refine the proposal a bit more.
I think that having an 8-byte EOS as 0x is OK -- I
think my concern about backwards compatibility is unwarranted.
We also previously added the EOS to the File Format
https://github.com/apache/arrow/blob/master/do
Hmm... not sure about that. IMHO, if the old format is detected, then a
4-byte EOS marker should be used. If the new format is detected, then a
8-byte EOS marker should be used.
Regards
Antoine.
Le 07/08/2019 à 16:16, Wes McKinney a écrit :
> You make a good point. For backward compatibilit
You make a good point. For backward compatibility reasons, bytes 5
through 8 would need to be unspecified padding bytes, I think. Does
that sound right?
On Wed, Aug 7, 2019 at 9:02 AM Antoine Pitrou wrote:
>
>
> This may be coming a bit late, but I realize we could take the
> opportunity to *also
This may be coming a bit late, but I realize we could take the
opportunity to *also* make the end-of-stream marker a 8-bytes marker
(rather than 4-bytes). What do you think?
Regards
Antoine.
Le 06/08/2019 à 22:15, Wes McKinney a écrit :
> hi all,
>
> As we've been discussing for the last 5
hi all,
As we've been discussing for the last 5 weeks or so [1], there is a
need to introduce 4 bytes of padding into the preamble of the
"encapsulated IPC message" format to ensure that the Flatbuffers
metadata payload begins on an 8-byte aligned memory offset. The
alternative to this would be fo
34 matches
Mail list logo
