Re: Using custom class as a key for groupByKey() or reduceByKey()

2014-06-15 Thread Andrew Ash
Good point Sean. I've filed a ticket to document the equals() / hashCode() requirements for custom keys in the Spark documentation, as this has come up a few times on the user@ list. https://issues.apache.org/jira/browse/SPARK-2148 On Sun, Jun 15, 2014 at 12:11 PM, Sean Owen wrote: > In Java

Re: Using custom class as a key for groupByKey() or reduceByKey()

2014-06-15 Thread Sean Owen
In Java at large, you must always implement hashCode() when you implement equals(). This is not specific to Spark. This is to maintain the contract that two equals() instances have the same hash code, and that's not the case for your class now. This causes weird things to happen wherever the hash c