After doing a groupBy operation, I have the following result:
val res =
("ID1",ArrayBuffer((1458046000001,"ID1","japan")))
("ID3",ArrayBuffer((1458650800000,"ID3","canada"),
(1458996400000,"ID3","china")))
("ID2",ArrayBuffer((1457527600000,"ID2","usa"),
(1459342000000,"ID2","usa")))
Now I need to output for each group, the size of each group and the max of
the first field, which is a timestamp.
So, I tried the following:
1) res.map(group => (group._2.size, group._2._1.max))
But I got an error : value _1 is not a member of Iterable[(Long, String,
String)]
2) I also tried: res.map(group => (group._2.size, group._2[1].max)), but got
an error for that as well.
What is the right way to get the max of the timestamp field (the first field
in the ArrayBuffer) for each group?
thanks.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/groupBy-question-tp7357.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
