We're close to having the integration tests all passing -- Julien and I have been hammering out the lingering nuances between the Java and C++ implementations. There's a number of JIRAs remaining linked to from this issue:
https://github.com/apache/arrow/pull/219 On Mon, Nov 21, 2016 at 8:55 PM, Wes McKinney <wesmck...@gmail.com> wrote: > hey Ted > > On Mon, Nov 21, 2016 at 8:20 PM, Ted Dunning <ted.dunn...@gmail.com> wrote: >> Wes, >> >> This is awesome. >> >> Does it, however, imply that to run the tests that a C programmer will need >> a working Java environment and a Java programmer will need a C environment? >> >> Is there any way around that? Possibly by storing golden bits for the >> in-memory images somewhere? >> > > Easiest thing would be to create a Dockerfile for experimentation -- > this would be useful for benchmarking on different hardware > environments as well. We'll want to run the integration tests either > in Travis CI or Circle CI anyway (right now we have the Java and > C++/Python unit tests running in separate build setups in Travis CI), > so it hopefully wouldn't be a great deal of additional effort to put > everything into a container recipe. > > Wes > >> >> >> >> >> On Mon, Nov 21, 2016 at 2:05 PM, Wes McKinney <wesmck...@gmail.com> wrote: >> >>> hi folks, >>> >>> After a long road, we're getting very close to having tests proving >>> that the Java and C++ Arrow implementations are binary compatible -- >>> this will be an exciting major milestone for the project. If you >>> haven't been following along recent JIRAs, the way these tests work is >>> as follows: >>> >>> 1) Testing dataset is specified in JSON format >>> >>> 2) Producer library (e.g. Java) reads JSON into Arrow in-memory, then >>> writes out to an Arrow file IPC binary format >>> >>> 3) Consumer library (e.g. C++) attempts to read both the JSON and the >>> binary file yielded by the producer library. The consumer compares the >>> in-memory schemas and columnar data structures and indicates whether >>> they are binary-identical >>> >>> I found a couple initial incompatibilities in the file format >>> implementations, cited here: >>> https://github.com/apache/arrow/pull/211#issuecomment-262080545. >>> >>> Thanks >>> Wes >>>
