RE: Appending to streaming file format

2018-05-14 Thread Ambalu, Robert
Will do, thx -Original Message- From: Antoine Pitrou [mailto:anto...@python.org] Sent: Monday, May 14, 2018 11:18 AM To: dev@arrow.apache.org Subject: Re: Appending to streaming file format Le 14/05/2018 à 17:17, Ambalu, Robert a écrit : > Cool, thanks Antoine. So this fixes be

Re: Appending to streaming file format

2018-05-14 Thread Antoine Pitrou
is to open an issue (GitHub or JIRA) and someone will investigate. Regards Antoine. > > -Original Message- > From: Antoine Pitrou [mailto:anto...@python.org] > Sent: Monday, May 14, 2018 11:07 AM > To: dev@arrow.apache.org > Subject: Re: Appending to streaming file form

RE: Appending to streaming file format

2018-05-14 Thread Ambalu, Robert
To: dev@arrow.apache.org Subject: Re: Appending to streaming file format Le 14/05/2018 à 16:37, Ambalu, Robert a écrit : > > Also, fyi, I opened a ticket last week that append is broken with the > FileOutputStream ( unrelated to this email thread ) > https://urldefense.proofpoint.

Re: Appending to streaming file format

2018-05-14 Thread Antoine Pitrou
Le 14/05/2018 à 16:37, Ambalu, Robert a écrit : > > Also, fyi, I opened a ticket last week that append is broken with the > FileOutputStream ( unrelated to this email thread ) > https://github.com/apache/arrow/issues/2018 Sorry, I hadn't seen your ticket (if you have found an actual bug, it's p

Appending to streaming file format

2018-05-14 Thread Ambalu, Robert
Hey, as far as I can tell it looks like appending to a streaming file format isn't currently supported, is that right? RecordBatchStreamWriter always writes the schema up front, and it doesn't look like a schema is expected mid file ( assuming im doing this append test correctly, this is the err