I have an RDD containing elements sorted in certain order. I would like to
map over the elements knowing the values of their respective previous and
next elements.
With regular List, I used to do this: ("input" is a List below)
// The first of the previous measures and the last of the next measures are
dummy
val dummy = new Measure()
val ml = (dummy :: input) :+ dummy
// Take 3 element at the time. Then 1st is the previous, the middle is the
current, and
// the last is the next.
ml.iterator.sliding(3).map( tri => produceMeasure(tri.head, tri.tail.head,
tri.last) ) toList
Now, in RDD, how do I do that (elegantly I hope)? I thought about zipping
an RDD with shifted self, but there are some messy partition issues about
zip.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Finding-previous-and-next-element-in-a-sorted-RDD-tp12621.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
