adriangb commented on code in PR #17273:
URL: https://github.com/apache/datafusion/pull/17273#discussion_r2292057562
##########
datafusion/core/src/datasource/listing/table.rs:
##########
@@ -1195,21 +1211,36 @@ impl TableProvider for ListingTable {
// if no files need to be read, return an `EmptyExec`
if partitioned_file_lists.is_empty() {
- let projected_schema = project_schema(&self.schema(), projection)?;
- return Ok(Arc::new(EmptyExec::new(projected_schema)));
+ let projected_schema = project_schema(&self.schema(),
projection.as_ref())?;
+ return Ok(ScanResult::new(
+ Arc::new(EmptyExec::new(projected_schema)),
+ filters.clone(),
+ ));
}
- let output_ordering = self.try_create_output_ordering()?;
+ let known_file_ordering = self.try_create_output_ordering()?;
+ let desired_file_ordering = match args.preferred_ordering() {
+ Some(ordering) if !ordering.is_empty() => {
+ // Prefer the ordering requested by the query to any inherint
file ordering
+ create_ordering(&self.table_schema, &[ordering.to_vec()])?
+ .first()
+ .cloned()
+ }
+ Some(_) | None => {
+ // If the query did not request a specific ordering, fall back
to any inherent file ordering
+ known_file_ordering.first().cloned()
+ }
+ };
Review Comment:
Not too sure about correctness here. The `Vec<Vec<SortExpr>>` is a bit
wonky. I think the answer is to add tests / benchmarks proving that the
"correct" ordering for files is being chosen.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
