[
https://issues.apache.org/jira/browse/SPARK-17896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15703320#comment-15703320
]
Andrew Ray commented on SPARK-17896:
------------------------------------
The given code seems to work in 2.0.2
> Dataset groupByKey + reduceGroups fails with codegen-related exception
> ----------------------------------------------------------------------
>
> Key: SPARK-17896
> URL: https://issues.apache.org/jira/browse/SPARK-17896
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.0.1
> Environment: Databricks, MacOS
> Reporter: Adam Breindel
>
> possible regression: works on 2.0, fails on 2.0.1
> following code raises exception related to wholestage codegen:
> case class Zip(city:String, zip:String, state:String)
> val z1 = Zip("New York", "10000", "NY")
> val z2 = Zip("New York", "10001", "NY")
> val z3 = Zip("Chicago", "60606", "IL")
> val zips = sc.parallelize(Seq(z1, z2, z3)).toDS
> zips.groupByKey(_.state).reduceGroups((z1, z2) => Zip("*", z1.zip + " " +
> z2.zip, z1.state)).show
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
