We just got the integration test suite (binary compatibility between Java and C++) passing in Travis CI today!
https://travis-ci.org/wesm/arrow/builds/182725476 Big team effort, congrats on all the hard work! On Fri, Dec 2, 2016 at 10:50 AM, Wes McKinney <wesmck...@gmail.com> wrote: > We're close to having the integration tests all passing -- Julien and > I have been hammering out the lingering nuances between the Java and > C++ implementations. There's a number of JIRAs remaining linked to > from this issue: > > https://github.com/apache/arrow/pull/219 > > On Mon, Nov 21, 2016 at 8:55 PM, Wes McKinney <wesmck...@gmail.com> wrote: >> hey Ted >> >> On Mon, Nov 21, 2016 at 8:20 PM, Ted Dunning <ted.dunn...@gmail.com> wrote: >>> Wes, >>> >>> This is awesome. >>> >>> Does it, however, imply that to run the tests that a C programmer will need >>> a working Java environment and a Java programmer will need a C environment? >>> >>> Is there any way around that? Possibly by storing golden bits for the >>> in-memory images somewhere? >>> >> >> Easiest thing would be to create a Dockerfile for experimentation -- >> this would be useful for benchmarking on different hardware >> environments as well. We'll want to run the integration tests either >> in Travis CI or Circle CI anyway (right now we have the Java and >> C++/Python unit tests running in separate build setups in Travis CI), >> so it hopefully wouldn't be a great deal of additional effort to put >> everything into a container recipe. >> >> Wes >> >>> >>> >>> >>> >>> On Mon, Nov 21, 2016 at 2:05 PM, Wes McKinney <wesmck...@gmail.com> wrote: >>> >>>> hi folks, >>>> >>>> After a long road, we're getting very close to having tests proving >>>> that the Java and C++ Arrow implementations are binary compatible -- >>>> this will be an exciting major milestone for the project. If you >>>> haven't been following along recent JIRAs, the way these tests work is >>>> as follows: >>>> >>>> 1) Testing dataset is specified in JSON format >>>> >>>> 2) Producer library (e.g. Java) reads JSON into Arrow in-memory, then >>>> writes out to an Arrow file IPC binary format >>>> >>>> 3) Consumer library (e.g. C++) attempts to read both the JSON and the >>>> binary file yielded by the producer library. The consumer compares the >>>> in-memory schemas and columnar data structures and indicates whether >>>> they are binary-identical >>>> >>>> I found a couple initial incompatibilities in the file format >>>> implementations, cited here: >>>> https://github.com/apache/arrow/pull/211#issuecomment-262080545. >>>> >>>> Thanks >>>> Wes >>>>
