Thanks Nicholas. The obvious fix for this issue, in my case, was to cache the input since it's only 35 megabytes.
Dan On Wed, Jun 4, 2014 at 7:34 AM, Nicholas Chammas <[email protected] > wrote: > I think by default a thread can die up to 4 times before Spark considers > it a failure. Are you seeing that happen? I believe that is a configurable > thing, but don't know off the top of my head how to change it. > > I've seen this error before when reading data from a large amount of files > on S3, and it is typically harmless. Spark just retries the operation and > proceeds normally. > > > On Wed, Jun 4, 2014 at 4:05 AM, yuzeh <[email protected]> wrote: > >> I should add that I'm using spark 0.9.1. >> >> Thanks! >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/SocketException-when-reading-from-S3-s3n-format-tp6889p6890.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> > >
