andygrove commented on PR #1904: URL: https://github.com/apache/datafusion-comet/pull/1904#issuecomment-2985784145
> > @comphead we seem to have a correctness issue when enabling LeftAnti + BuildRight: > > ``` > > [info] +- == Initial Plan == > > [info] CometBroadcastHashJoin [c1#253], [c2#260], LeftAnti, BuildRight > > [info] :- CometScan parquet spark_catalog.default.test_table[c1#253,c2#254] Batched: true, DataFilters: [], Format: CometParquet, Location: InMemoryFileIndex(1 paths)[file:/home/andy/git/apache/apache-spark-3.5.6/sql/core/spark-warehouse..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<c1:int,c2:int> > > [info] +- CometBroadcastExchange [c2#260] > > [info] +- CometScan parquet spark_catalog.default.test_table[c2#260] Batched: true, DataFilters: [], Format: CometParquet, Location: InMemoryFileIndex(1 paths)[file:/home/andy/git/apache/apache-spark-3.5.6/sql/core/spark-warehouse..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<c2:int> > > [info] > > [info] == Results == > > [info] > > [info] == Results == > > [info] !== Correct Answer - 0 == == Spark Answer - 2 == > > [info] !struct<> struct<c1:int,c2:int> > > [info] ! [4,null] > > [info] ! [null,0] (QueryTest.scala:244) > > ``` > > Thanks @andygrove for experimenting with it, please correct me if I'm wrong. The result shows correctness issue but before it has been failing on not supported error? Previously, we just had a check in Comet and fell back to Spark for this type of join, therefore avoiding the issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
