[
https://issues.apache.org/jira/browse/SPARK-18134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677266#comment-15677266
]
Herman van Hovell commented on SPARK-18134:
-------------------------------------------
There is not a political reason for doing this. The is reason is that a
dataType needs to be comparable if you want to use it in any kind of hashing or
joining; a map by definition is not comparable, it only enforces that its keys
need to be unique.
The only practical solution here would be to add a ordered flag to map type,
and that we sort all maps as soon as they are used in keys.
Would you be willing to work on this?
> SQL: MapType in Group BY and Joins not working
> ----------------------------------------------
>
> Key: SPARK-18134
> URL: https://issues.apache.org/jira/browse/SPARK-18134
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.5.0, 1.5.1, 1.5.2, 1.6.0, 1.6.1, 1.6.2, 2.0.0, 2.0.1
> Reporter: Christian Zorneck
>
> Since version 1.5 and issue SPARK-9415, MapTypes can no longer be used in
> GROUP BY and join clauses. This makes it incompatible to HiveQL. So, a Hive
> feature was removed from Spark. This makes Spark incompatible to various
> HiveQL statements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
