1. Caused by: java.io.IOException: Failed to connect to
ip-10-12-46-235.us-west-2.compute.internal/10.12.46.235:55681
2. at
org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:216)
3. at
org.apache.spark.network.client.
I'm not quite sure how this is a memory problem. There are no OOM
exceptions and the job only breaks when actions are ran in parallel,
submitted to the scheduler by different threads.
The issue is that the doGetRemote function does not retry when it is denied
access to a cache block.
On May 8, 201
Brandon,
how much memory are you giving to your executors - did you check if there
were dead executors in your application logs.. Most likely you require
higher memory for executors..
Ashish
On Sun, May 8, 2016 at 1:01 PM, Brandon White
wrote:
> Hello all,
>
> I am running a Spark application
Hello all,
I am running a Spark application which schedules multiple Spark jobs.
Something like:
val df = sqlContext.read.parquet("/path/to/file")
filterExpressions.par.foreach { expression =>
df.filter(expression).count()
}
When the block manager fails to fetch a block, it throws an excepti
