I second Weston's comments. The idea of separate files is part of the de
jure spec but not the de facto one. It's up to the parquet community
whether the de facto spec should be "altered" . Afaik, zero oss readers
support use of this field.
On Wed, May 18, 2022, 8:53 AM Weston Pace wrote:
> I
I can try and clarify my earlier feedback:
This is an Arrow datasets question if your goal is to create multiple
independent parquet files, each one a complete file, and read them as
a combined dataset.
This is not an Arrow question (but instead a parquet question) if your
goal is to create a sing
