On Mon, 12 Jun 2023 at 21:30, Jerald Alex wrote:
>
> hi Weston,
>
> Thank you so much for taking the time to respond. Really appreciate it.
>
> I'm using parquet files. So would it be possible to elaborate the below.? I
> cannot seem to find any documentation for ParquetFileFragment.
>
> "there ma
hi Weston,
Thank you so much for taking the time to respond. Really appreciate it.
I'm using parquet files. So would it be possible to elaborate the below.? I
cannot seem to find any documentation for ParquetFileFragment.
"there may even be a way to skip row groups by creating a fragment per row
> I would like to know if it is possible to skip the specific set of
batches,
> for example, the first 10 batches and read from the 11th Batch.
This sort of API does not exist today. You can skip files by making a
smaller dataset with fewer files (and I think, with parquet, there may even
be a wa
