Smallfu666 commented on PR #6178: URL: https://github.com/apache/datafusion-comet/pull/6178#issuecomment-5977409025
Thanks for taking a closer look! 1. Done. I narrowed the guard so it only refuses when the kernel output contains a map. The cast-to-string case now stays in the dispatcher, and I added the `exists(map_keys(...))` case as well. Both match Spark. 2. Good catch. I reproduced the constant-folding case: before the fix, Comet returns `[1, 0]` while Spark returns `[1, NULL]`. The guard now also catches map literals containing a null key, including nested ones. The SQL test suite disables `ConstantFolding`, so I put the end-to-end regression test in `CometCodegenSuite`, where the cast is actually folded. 3. I tried the decimal case, but Spark rejects it during analysis with `DATATYPE_MISMATCH.CAST_WITHOUT_SUGGESTION`. For TRY casts on map keys, `canTryCast` requires the key conversion not to be `forceNullable`, so a narrowing decimal key cast cannot reach this path. I checked this on the supported Spark versions from 3.4 through 4.2. I also removed the overly broad wording about Spark storing the default key value from the PR description. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
