masonh22 opened a new issue, #17756:
URL: https://github.com/apache/datafusion/issues/17756
### Describe the bug
In the push-down filter optimization for window operators, we create a set
of all the columns that appear in the window "partition by" expression and
check whether all columns in a filter are contained in the set of "partition
by" columns. If so, that filter can be pushed down past the window operator.
The problem is with how columns are extracted from the "partition by"
expression. Currently, the code calls `Expr::schema_name()` to serialize the
column name to a string, then parses it into a column using
`Column::from_qualified_name()`. The problem is that if
`Column::from_qualified_name()` gets a column name that can't be parsed by the
rules of `GenericDialect`, it throws the whole string into the `name` field,
ignoring the relation.
For example, if we have a partition column that looks something like `Column
{ relation: Some(Bare { table: "test" }), name: "$a", .. }`, the window push
down rule will extract this into `Column { relation: None, name: "test.$a", ..
}`. Then, the logic will falsely identify that a filter containing only the
column "test.$a" can't be pushed down because it's represented differently than
the column we extracted.
### To Reproduce
Create a table with field names that start with a '$', create a logical plan
with a window and filter operation that should be pushed down, and inspect the
optimized plan.
### Expected behavior
The filter should be pushed down.
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
