viirya commented on PR #6726: URL: https://github.com/apache/datafusion-comet/pull/6726#issuecomment-6609891497
Thanks for adding the collection-bearing join, that covers it. The fork CI's Spark 4.1 exec job has one failure on `26429c793`: `CometInMemoryCacheSuite` "AQE builds a cache whose final stage ends in a Spark operator straight into Arrow". #6991 added `CometCacheBuildRule`, which also asks `CometSparkToColumnarExec.isSchemaSupported` whether it can replace the root `RowToColumnarExec` of a cache build. Its `array of int` case expects no conversion, and this PR converts `array<int>`, so the plan now has a `CometSparkRowToColumnar`. Could that case expect a conversion instead? The test already compares the cached rows with Spark's answer, so it would cover arrays on the cache build path too. A type this PR still declines, such as a calendar interval, could take over as the case that stays on Spark's transition. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
