chenkovsky commented on PR #14057: URL: https://github.com/apache/datafusion/pull/14057#issuecomment-2594790967
> > > Can these metadata columns utilize normal column properties, like ordering equivalences, constantness, distinctness etc.? For example, AFAIU rowid is an ordered column, and if I sort the table by rowid, the SortExec would be removed? (it seems to me not yet at this point) Can we iterate over the design to support those capabilities, too? > > > > > > I with this PR a custom table provider that was ordered by row_id could communicate that information to avoid a SortExec > > From what I can tell, the metadata columns is only a notion in the `LogicalPlan` > > Specifically, the `ExecutionPlan` returned by the provider is no different than any other `ExecutionPlan` so it can communicate sortedness via `ExecutionPlan::properties` as normal > > What I mean is: > > https://github.com/chenkovsky/datafusion/blob/5c4b5c4c7aee47b6287e5fcf32d87485ee1c9e37/datafusion/core/tests/sql/metadata_columns.rs#L389 > > When I print this query, there exists a SortExec for _rowid. But what I understand is _rowid should be a one-by-one increasing column? Maybe not, I use vec to store values in test, but if the inner datastructure is btree, the scan order is not always increasing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
