Re: Read Arrow 0.9.0 output using newer pyarrow version

2019-05-13 Thread Wes McKinney
hi Rares, Like I said the files should be forward compatible. Can you open a JIRA issue and give code to reproduce the issue? Thanks On Mon, May 13, 2019 at 7:43 AM Rares Vernica wrote: > > Hi Wes, > > Thanks for your answer. I finally got to test this out. To recap, I'm > writing Arrow files f

Re: Read Arrow 0.9.0 output using newer pyarrow version

2019-05-13 Thread Rares Vernica
Hi Wes, Thanks for your answer. I finally got to test this out. To recap, I'm writing Arrow files from C++ using Arrow 0.9.0. Then, I'm trying to read these files from Python. I tried Python 2.7.15 and PyArrow 0.10.0 to 0.13.0. In all these cases I get an error. (PyArrow 0.9.0 works fine, as expe

Re: Read Arrow 0.9.0 output using newer pyarrow version

2019-03-11 Thread Wes McKinney
hi Rares -- IPC messages produced by 0.9.0 should be forward compatible. I opened https://issues.apache.org/jira/browse/ARROW-921 some time ago about adding some tools to integration test one version versus another to obtain hard proof of this, but this work has not been completed yet (any takers?)

Read Arrow 0.9.0 output using newer pyarrow version

2019-03-10 Thread Rares Vernica
Hello, I have a C++ library using Arrow 0.9.0 to serialize data The code looks like this: std::shared_ptr arrowBatch; arrowBatch = arrow::RecordBatch::Make(_arrowSchema, nCells, _arrowArrays); std::shared_ptr arrowBuffer(new arrow::PoolBuffer(_arrowPool)); arrow::io::BufferOutputStream arrowStre