Maybe using mapPartitions and .sequence inside it? El 26/7/2015 10:22 p. m., "Ayoub" <benali.ayoub.i...@gmail.com> escribió:
> Hello, > > I am trying to convert the result I get after doing some async IO : > > val rdd: RDD[T] = // some rdd > > val result: RDD[Future[T]] = rdd.map(httpCall) > > Is there a way collect all futures once they are completed in a *non > blocking* (i.e. without scala.concurrent > Await) and lazy way? > > If the RDD was a standard scala collection then calling > "scala.concurrent.Future.sequence" would have resolved the issue but RDD is > not a TraversableOnce (which is required by the method). > > Is there a way to do this kind of transformation with an RDD[Future[T]] ? > > Thanks, > Ayoub. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/RDD-Future-T-Future-RDD-T-tp24000.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
