Using custom class as a key for groupByKey() or reduceByKey()

[Gaurav Jain]

I have a simple Java class as follows, that I want to use as a key while
applying groupByKey or reduceByKey functions:

private static class FlowId {
                public String dcxId;
                public String trxId;
                public String msgType;
                
                public FlowId(String dcxId, String trxId, String msgType) {
                        this.dcxId = dcxId;
                        this.trxId = trxId;
                        this.msgType = msgType;
                }
                
                public boolean equals(Object other) {
                        if (other == this) return true;
                        if (other == null) return false;
                        if (getClass() != other.getClass()) return false;
                        FlowId fid = (FlowId) other;
                        if (this.dcxId.equals(fid.dcxId) && 
this.trxId.equals(fid.trxId) &&
                                        this.msgType.equals(fid.msgType)) {
                                return true;
                        }
                        return false;
                }
}

I figured that an equals() method would need to be overridden to ensure
comparison of keys, but still entries with the same key are listed
separately after applying a groupByKey(), for example. What further
modifications are necessary to enable usage of above class as a key. Right
now, I have fallen back to using Tuple3<String, String, String> instead of
the FlowId class, but it makes the code unnecessarily verbose.



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/Using-custom-class-as-a-key-for-groupByKey-or-reduceByKey-tp7640.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.