Understood, thank you for the quick response
-Original Message-
From: Micah Kornfield
Sent: Wednesday, June 16, 2021 4:13 PM
To: dev
Cc: Shamis, Michael
Subject: Re: [C++] Parquet streaming
Correct, you cannot recover a partially written parquet file.
This is only really feasible
Correct, you cannot recover a partially written parquet file.
This is only really feasible with the Arrow Streaming format and even there
you might run into issues if the data is not synced at the appropriate
place. The arrow file format requires a footer be written so it has the
same issue.
-Mi
Apache community, I just want to confirm my understanding of parquet files.
I have a streaming set of data that may be produced in realtime. Ideally I
would stream it into a parquet file ( and if the process crashes, still be able
to read some part of what was streamed ).
I can do this with arro
