jonahgao opened a new issue, #14303: URL: https://github.com/apache/datafusion/issues/14303
### Describe the bug I found that `UnwrapCastInComparison` always assumes the cast operation can succeed, but when it cannot, it results in incorrect optimization results. ### To Reproduce Run query in CLI (compiled from the latest main: https://github.com/apache/datafusion/commit/f77579108d1dc0285636fbfb24507d2bfca66446) ```sh DataFusion CLI v44.0.0 > with t as (select 1000000 as a) select try_cast(a as smallint) > 1 from t; +----------------+ | t.a > Int64(1) | +----------------+ | true | +----------------+ 1 row(s) fetched. Elapsed 0.008 seconds. > with t as (select 1000000 as a) select cast(a as smallint) > 1 from t; +----------------+ | t.a > Int64(1) | +----------------+ | true | +----------------+ 1 row(s) fetched. Elapsed 0.007 seconds. ``` ### Expected behavior When optimizations are disabled, the above queries will produce different results, which are correct. ```sh > set datafusion.optimizer.max_passes=0; 0 row(s) fetched. Elapsed 0.003 seconds. > with t as (select 1000000 as a) select try_cast(a as smallint) > 1 from t; +----------------+ | t.a > Int64(1) | +----------------+ | NULL | +----------------+ 1 row(s) fetched. Elapsed 0.006 seconds. > with t as (select 1000000 as a) select cast(a as smallint) > 1 from t; Arrow error: Cast error: Can't cast value 1000000 to type Int16 ``` ### Additional context I don't think this is a very urgent bug because both Spark and DuckDB have similar issues. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
