PedroMDuarte commented on issue #485: URL: https://github.com/apache/datafusion-comet/issues/485#issuecomment-2143904091
Is it possible that the difference report is not sorting the data in the same way for spark and comet? I'm surprised by the discrepancy: ``` Spark: [1.584962500721156] Comet: [-Infinity] ``` the 1.58 value is log2(3) whereas -Infinity is log2(0). Running without and with comet enabled I do see a discrepancy for log2(0) as follows: ``` spark +---+------------------+-----------------+ | a| LOG2(a)|(LOG2(a) IS NULL)| +---+------------------+-----------------+ | 0| null| true| | 1| 0.0| false| | 2| 1.0| false| | 3| 1.584962500721156| false| | 4| 2.0| false| | 5| 2.321928094887362| false| | 6| 2.584962500721156| false| | 7| 2.807354922057604| false| | 8| 3.0| false| | 9|3.1699250014423126| false| +---+------------------+-----------------+ ``` ``` spark with comet +---+-----------------+-----------------+ | a| LOG2(a)|(LOG2(a) IS NULL)| +---+-----------------+-----------------+ | 0| -Infinity| false| | 1| 0.0| false| | 2| 1.0| false| | 3|1.584962500721156| false| | 4| 2.0| false| | 5|2.321928094887362| false| | 6|2.584962500721156| false| | 7|2.807354922057604| false| | 8| 3.0| false| | 9|3.169925001442312| false| +---+-----------------+-----------------+ ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
