Re: [I] Add support for `MapSort` expression in Spark 4.0.0 [datafusion-comet]

2025-07-25 Thread via GitHub
parthchandra commented on issue #1941: URL: https://github.com/apache/datafusion-comet/issues/1941#issuecomment-3120675532 Oh well. For Maps, you might want to look at the mapping of Spark's MapType to a Parquet Schema here: https://github.com/apache/datafusion-comet/blob/320ce55eec

Re: [I] Add support for `MapSort` expression in Spark 4.0.0 [datafusion-comet]

2025-07-25 Thread via GitHub
rishvin commented on issue #1941: URL: https://github.com/apache/datafusion-comet/issues/1941#issuecomment-3120621876 > > managed to scan the map-type by setting `CometConf.COMET_NATIVE_SCAN_IMPL.key -> native_datafusion `. Added `map_sort` UDF with return type as `Map`. > > Right.

Re: [I] Add support for `MapSort` expression in Spark 4.0.0 [datafusion-comet]

2025-07-24 Thread via GitHub
parthchandra commented on issue #1941: URL: https://github.com/apache/datafusion-comet/issues/1941#issuecomment-3114897910 > managed to scan the map-type by setting `CometConf.COMET_NATIVE_SCAN_IMPL.key -> native_datafusion `. Added `map_sort` UDF with return type as `Map`. Right. `

Re: [I] Add support for `MapSort` expression in Spark 4.0.0 [datafusion-comet]

2025-07-23 Thread via GitHub
rishvin commented on issue #1941: URL: https://github.com/apache/datafusion-comet/issues/1941#issuecomment-3111745019 > Some Updates: I have a simple test to start with, which will produce `_groupingmapsort`. > > ``` > val data = Seq( > | Map("a" -> 1, "b" -> 2), >

Re: [I] Add support for `MapSort` expression in Spark 4.0.0 [datafusion-comet]

2025-07-21 Thread via GitHub
rishvin commented on issue #1941: URL: https://github.com/apache/datafusion-comet/issues/1941#issuecomment-3101303544 Some Updates: I have a simple test to start with, which will produce `_groupingmapsort`. ``` val data = Seq( | Map("a" -> 1, "b" -> 2), | Map("a"

Re: [I] Add support for `MapSort` expression in Spark 4.0.0 [datafusion-comet]

2025-07-13 Thread via GitHub
rishvin commented on issue #1941: URL: https://github.com/apache/datafusion-comet/issues/1941#issuecomment-3067159959 I would like to work on this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t