Thanks. As my files are defined to be non-splittable, I eventually I ended up
using mapPartitionsWithIndex() taking the split ID as index
def g(splitIndex, iterator):
yield (splitIndex, iterator.next())
myRDD.mapPartitionsWithIndex(g)
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/RDD-Row-Index-tp12457p12550.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
