alamb commented on PR #13933: URL: https://github.com/apache/datafusion/pull/13933#issuecomment-2565321646
Hi @zhuqi-lucas -- sorry if we caused confusion here. I agree with @berkaysynnada and @ozankabak that ordering information is already represented in plans using [`EquivalenceProperties`](https://docs.rs/datafusion/latest/datafusion/physical_expr/struct.EquivalenceProperties.html) - As I understand https://github.com/apache/datafusion/issues/13891: The idea is to store the output sorted order in the parquet file So this would look something like: 1. `COPY (SELECT ... ORDER BY x, y) to 'foo.parquet'` 2. `foo.parquet` has some new metadata like `DATAFUSION_ORDER_BY = `x ASC, y ASC` 3. On the next `SELECT ... FROM 'foo.parquet'` the ParquetExec would look for the ` DATAFUSION_ORDER_BY` metadata in the file and if present update the `EquivalenceProperties` to reflect an ordering `x, y` Does that make sense? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
