David Mollitor created SPARK-59413:
--------------------------------------
Summary: Treat the Collate passthrough as cheap in
CollapseProject.isCheap
Key: SPARK-59413
URL: https://issues.apache.org/jira/browse/SPARK-59413
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.1.0
Reporter: David Mollitor
h2. Summary
{{CollapseProject.isCheap(e: Expression)}} decides whether an expression is
cheap enough to duplicate/inline. It currently returns true for {{Attribute}} /
{{OuterReference}} /
{{BoundReference}}, foldable expressions, some {{PythonUDF}}s, and {{Alias}} /
{{ExtractValue}} over cheap children -- but it does not recognize the
{{Collate}} expression.
{{Collate}} (the SQL {{collate(expr, 'COLLATION')}} function / {{expr COLLATE
COLLATION}} syntax) is a pure
[pass-through|https://github.com/apache/spark/blob/0cfb8bfca7d87769b618b43010da8f50c0b62218/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala#L69-L74]:
its {{eval}} delegates directly to the child ({{child.eval(row)}}), its
codegen delegates to the child ({{child.genCode(ctx)}}), and it never evaluates
its collation argument. It performs no per-row work -- it only re-tags the
collation carried in the result _type_. It is deterministic and, at runtime,
exactly as cheap as the child it wraps.
Because {{isCheap}} does not know this, {{Collate(cheapChild)}} is treated as
non-cheap. That needlessly blocks safe duplication/inlining of collated columns
across every {{isCheap}}-gated rule (CollapseProject inlining, the multi-LIKE
{{LikeSimplification}} rules, the {{FilterExec}} CSE gate,
{{RewriteWithExpression}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
