This is great, nice job guys! On Fri, Dec 9, 2016 at 2:15 PM, Jason Altekruse <altekruseja...@gmail.com> wrote:
> Congrats guys, great work! > > On Fri, Dec 9, 2016 at 2:14 PM, Julien Le Dem <jul...@dremio.com> wrote: > > > Woot! > > 🎉 > > > > On Fri, Dec 9, 2016 at 2:07 PM, Wes McKinney <wesmck...@gmail.com> > wrote: > > > > > We just got the integration test suite (binary compatibility between > > > Java and C++) passing in Travis CI today! > > > > > > https://travis-ci.org/wesm/arrow/builds/182725476 > > > > > > Big team effort, congrats on all the hard work! > > > > > > On Fri, Dec 2, 2016 at 10:50 AM, Wes McKinney <wesmck...@gmail.com> > > wrote: > > > > We're close to having the integration tests all passing -- Julien and > > > > I have been hammering out the lingering nuances between the Java and > > > > C++ implementations. There's a number of JIRAs remaining linked to > > > > from this issue: > > > > > > > > https://github.com/apache/arrow/pull/219 > > > > > > > > On Mon, Nov 21, 2016 at 8:55 PM, Wes McKinney <wesmck...@gmail.com> > > > wrote: > > > >> hey Ted > > > >> > > > >> On Mon, Nov 21, 2016 at 8:20 PM, Ted Dunning <ted.dunn...@gmail.com > > > > > wrote: > > > >>> Wes, > > > >>> > > > >>> This is awesome. > > > >>> > > > >>> Does it, however, imply that to run the tests that a C programmer > > will > > > need > > > >>> a working Java environment and a Java programmer will need a C > > > environment? > > > >>> > > > >>> Is there any way around that? Possibly by storing golden bits for > the > > > >>> in-memory images somewhere? > > > >>> > > > >> > > > >> Easiest thing would be to create a Dockerfile for experimentation -- > > > >> this would be useful for benchmarking on different hardware > > > >> environments as well. We'll want to run the integration tests either > > > >> in Travis CI or Circle CI anyway (right now we have the Java and > > > >> C++/Python unit tests running in separate build setups in Travis > CI), > > > >> so it hopefully wouldn't be a great deal of additional effort to put > > > >> everything into a container recipe. > > > >> > > > >> Wes > > > >> > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> On Mon, Nov 21, 2016 at 2:05 PM, Wes McKinney <wesmck...@gmail.com > > > > > wrote: > > > >>> > > > >>>> hi folks, > > > >>>> > > > >>>> After a long road, we're getting very close to having tests > proving > > > >>>> that the Java and C++ Arrow implementations are binary compatible > -- > > > >>>> this will be an exciting major milestone for the project. If you > > > >>>> haven't been following along recent JIRAs, the way these tests > work > > is > > > >>>> as follows: > > > >>>> > > > >>>> 1) Testing dataset is specified in JSON format > > > >>>> > > > >>>> 2) Producer library (e.g. Java) reads JSON into Arrow in-memory, > > then > > > >>>> writes out to an Arrow file IPC binary format > > > >>>> > > > >>>> 3) Consumer library (e.g. C++) attempts to read both the JSON and > > the > > > >>>> binary file yielded by the producer library. The consumer compares > > the > > > >>>> in-memory schemas and columnar data structures and indicates > whether > > > >>>> they are binary-identical > > > >>>> > > > >>>> I found a couple initial incompatibilities in the file format > > > >>>> implementations, cited here: > > > >>>> https://github.com/apache/arrow/pull/211#issuecomment-262080545. > > > >>>> > > > >>>> Thanks > > > >>>> Wes > > > >>>> > > > > > > > > > > > -- > > Julien > > >
