Juan, thanks for sharing this. I am facing what looks like a similar issue having to do with variable grouped upsampling (sampling some groups at different rates, sometimes > 100%). I will study the approach you took.
As for the topic of this thread, I think it is important to separate two issues: - Logical RDD-style operations on Iterables - Physical RDD-style operations on partitioned data Issues related to nested RDDs, jobs and the scheduler only apply to the latter unless we want to heavily optimize the performance of the former. I wouldn't do that until we see enough usage of the former to know what's worth optimizing. Thanks, Sim -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/RDD-API-patterns-tp14116p14193.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
