alamb commented on issue #14607: URL: https://github.com/apache/datafusion/issues/14607#issuecomment-2655004320
> Seems like the best way would be to configure FileGroupPartitioner through FileSource. The other option would be to make FileRange an enum, but that would still mean we (and any other format with a similar structure) will have to maintain our own repartitioning logic. I agree -- having a Format specific repartitioner makes the most sense Note sure if are following the latest developments on main, but there is the newly added `FileSource` triat https://github.com/apache/datafusion/blob/2d303346c7d44a78d317e7b7b959cbd94edfb891/datafusion/core/src/datasource/data_source.rs#L69 Maybe instead of `supports_repartition` we could extend it to directly do the repartitioning 🤔 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
