Thank you, Akhil. Actually the problem was solved last week and I did not
have time to report back. The error was caused by YARN killing the
container because executors use more off-heap memory that they were
assigned. There was nothing in the exectuor log, but the AM log clearly
states this is the
Can you look a bit deeper in the executor logs? It could be filling up the
memory and getting killed.
Thanks
Best Regards
On Mon, Oct 5, 2015 at 8:55 PM, Lan Jiang wrote:
> I am still facing this issue. Executor dies due to
>
> org.apache.avro.AvroRuntimeException: java.io.IOException: Filesyst
I am still facing this issue. Executor dies due to
org.apache.avro.AvroRuntimeException: java.io.IOException: Filesystem closed
at org.apache.avro.file.DataFileStream.hasNextBlock(DataFileStream.java:278)
at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:197)
at org.apache.avro.ma
Are you replicating any RDDs?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/java-io-IOException-Filesystem-closed-tp20150p21749.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
Does the sparkContext shuts down itself by default even if I dont mention
specifically in my code?? Because, I ran the application without
sc.context(), still I get file system closed error along with correct
output.
On Tue, Dec 2, 2014 at 2:20 PM, Akhil Das
wrote:
> It could be because those th
It could be because those threads are finishing quickly.
Thanks
Best Regards
On Tue, Dec 2, 2014 at 2:19 PM, rapelly kartheek
wrote:
> But, somehow, if I run this application for the second time, I find that
> the application gets executed and the results are out regardless of the
> same errors
But, somehow, if I run this application for the second time, I find that
the application gets executed and the results are out regardless of the
same errors in logs.
On Tue, Dec 2, 2014 at 2:08 PM, Akhil Das
wrote:
> Your code seems to have a lot of threads and i think you might be invoking
> sc
Your code seems to have a lot of threads and i think you might be invoking
sc.stop before those threads get finished.
Thanks
Best Regards
On Tue, Dec 2, 2014 at 12:04 PM, Akhil Das
wrote:
> What is the application that you are submitting? Looks like you might have
> invoked fs inside the app an
Sorry for the delayed response. Please find my application attached.
On Tue, Dec 2, 2014 at 12:04 PM, Akhil Das
wrote:
> What is the application that you are submitting? Looks like you might have
> invoked fs inside the app and then closed it within it.
>
> Thanks
> Best Regards
>
> On Tue, Dec
What is the application that you are submitting? Looks like you might have
invoked fs inside the app and then closed it within it.
Thanks
Best Regards
On Tue, Dec 2, 2014 at 11:59 AM, rapelly kartheek
wrote:
> Hi,
>
> I face the following exception when submit a spark application. The log
> fil
10 matches
Mail list logo
