Re: continuing processing when errors occur

2014-07-24 Thread Art Peel
Sorry, I sent this to the dev list instead of user. Please ignore. I'll re-post to the correct list. Regards, Art On Thu, Jul 24, 2014 at 11:09 AM, Art Peel wrote: > Our system works with RDDs generated from Hadoop files. It processes each > record in a Hadoop file and for a subset of those

continuing processing when errors occur

2014-07-24 Thread Art Peel
Our system works with RDDs generated from Hadoop files. It processes each record in a Hadoop file and for a subset of those records generates output that is written to an external system via RDD.foreach. There are no dependencies between the records that are processed. If writing to the external s