alamb commented on PR #14616: URL: https://github.com/apache/datafusion/pull/14616#issuecomment-2653423772
> `FileCompressionType`, `PartitionedFile`, `FileRange` can be move to `datasource`. > > If A and B is tightly couple, you need to pull partial structure out to C and import C for A and B. Not moving A and B together. The possible plan I proposed on https://github.com/apache/datafusion/issues/14444 proposes a structure like this - `datafusion-catalog-listing`: `ListingTable` and associated types like `PartitionedFile` - `datafusion-datasource-parquet`: `ParquetExec` and file firmat - `datafusion-datasource-avro` `AvroExec` and file formats - `datafusion-datasource-arrow` - `datafusion-datasource-json` - `datafusion-datasource-csv` This was before the refactor with `DataSource` landed: - https://github.com/apache/datafusion/pull/14224 @jayzhan211 are you proposing we add another crate in there, something like? - `datafusion-catalog-listing`: `ListingTable` and associated types like `PartitionedFile` - `datafusion-datasource`: **NEW** that holds `FileCompressionType`, `PartitonedFile`,and `DataSource` - `datafusion-datasource-parquet`: `ParquetExec` and file firmat - `datafusion-datasource-avro` `AvroExec` and file formats - `datafusion-datasource-arrow` - `datafusion-datasource-json` - `datafusion-datasource-csv` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
