[
https://issues.apache.org/jira/browse/SPARK-59044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59044:
-----------------------------------
Labels: optimizer pull-request-available sql (was: optimizer sql)
> Excluding EliminateOffsets causes OFFSET 0 to fail during physical planning
> ---------------------------------------------------------------------------
>
> Key: SPARK-59044
> URL: https://issues.apache.org/jira/browse/SPARK-59044
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 5.0.0
> Environment: Spark 5.0.0-SNAPSHOT, commit
> 9da9f8d673914d1648514f59d85e3adafb300d1a; macOS arm64; Java 17.0.17
> Reporter: Yibo Dong
> Priority: Major
> Labels: optimizer, pull-request-available, sql
>
> A legal OFFSET 0 query fails during physical planning when
> EliminateOffsets is excluded.
> h3. Reproduction
> {code:sql}
> SELECT 1 AS x OFFSET 0;
> {code}
> With the default optimizer configuration, the query executes successfully and
> returns:
> {code}
> 1
> {code}
> If only EliminateOffsets is excluded:
> {code}
> --conf
> spark.sql.optimizer.excludedRules=org.apache.spark.sql.catalyst.optimizer.EliminateOffsets
> {code}
> the same query fails during physical planning with:
> {code}
> org.apache.spark.SparkException: [INTERNAL_ERROR] The Spark SQL phase planning
> failed with an internal error.
> Caused by: java.lang.AssertionError: assertion failed
> at org.apache.spark.sql.execution.CollectLimitExec.<init>(limit.scala:49)
> {code}
> h3. Expected behavior
> OFFSET 0 is a no-op, so both configurations should execute successfully and
> return the same result:
> {code}
> 1
> {code}
> h3. Actual behavior
> With EliminateOffsets enabled, the query returns 1.
> When EliminateOffsets is excluded, the logical OFFSET 0 reaches physical
> planning and causes an assertion failure while constructing CollectLimitExec.
> h3. Additional information
> The only configuration change needed to reproduce the failure is excluding
> EliminateOffsets.
> This suggests that physical planning assumes OFFSET 0 has already been removed
> by EliminateOffsets, although the optimizer rule can currently be excluded.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
