zhuqi-lucas commented on PR #13933: URL: https://github.com/apache/datafusion/pull/13933#issuecomment-2567943900
> Hi @zhuqi-lucas -- sorry if we caused confusion here. I agree with @berkaysynnada and @ozankabak that ordering information is already represented in plans using [`EquivalenceProperties`](https://docs.rs/datafusion/latest/datafusion/physical_expr/struct.EquivalenceProperties.html) > > * As I understand [Find a way to communicate the ordering of a file back with the existing listing table implementation #13891](https://github.com/apache/datafusion/issues/13891): > > The idea is to store the output sorted order in the parquet file > > So this would look something like: > > 1. `COPY (SELECT ... ORDER BY x, y) to 'foo.parquet'` > 2. `foo.parquet` has some new metadata like `DATAFUSION_ORDER_BY = `x ASC, y ASC` > 3. On the next `SELECT ... FROM 'foo.parquet'` the ParquetExec would look for the ` DATAFUSION_ORDER_BY` metadata in the file and if present update the `EquivalenceProperties` to reflect an ordering `x, y` > > Does that make sense? Hi @alamb , the PR is ready for review now, i addressed the comments, also added unit testing and slt testing cases. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
