Thanks for putting this together, Andrew.
On Tue, Aug 12, 2014 at 2:11 AM, Andrew Ash wrote:
> Hi Chen,
>
> Please see the bug I filed at
> https://issues.apache.org/jira/browse/SPARK-2984 with the
> FileNotFoundException on _temporary directory issue.
>
> Andrew
>
>
> On Mon, Aug 11, 2014 at 1
Hi Chen,
Please see the bug I filed at
https://issues.apache.org/jira/browse/SPARK-2984 with the
FileNotFoundException on _temporary directory issue.
Andrew
On Mon, Aug 11, 2014 at 10:50 PM, Andrew Ash wrote:
> Not sure which stalled HDFS client issue your'e referring to, but there
> was one
Not sure which stalled HDFS client issue your'e referring to, but there was
one fixed in Spark 1.0.2 that could help you out --
https://github.com/apache/spark/pull/1409. I've still seen one related to
Configuration objects not being threadsafe though so you'd still need to
keep speculation on to
Andrew that is a good finding.
Yes, I have speculative execution turned on, becauseI saw tasks stalled on
HDFS client.
If I turned off speculative execution, is there a way to circumvent the
hanging task issue?
On Mon, Aug 11, 2014 at 11:13 AM, Andrew Ash wrote:
> I've also been seeing simil
I've also been seeing similar stacktraces on Spark core (not streaming) and
have a theory it's related to spark.speculation being turned on. Do you
have that enabled by chance?
On Mon, Aug 11, 2014 at 8:10 AM, Chen Song wrote:
> Bill
>
> Did you get this resolved somehow? Anyone has any insigh
Bill
Did you get this resolved somehow? Anyone has any insight into this problem?
Chen
On Mon, Aug 11, 2014 at 10:30 AM, Chen Song wrote:
> The exception was thrown out in application master(spark streaming driver)
> and the job shut down after this exception.
>
>
> On Mon, Aug 11, 2014 at 10
The exception was thrown out in application master(spark streaming driver)
and the job shut down after this exception.
On Mon, Aug 11, 2014 at 10:29 AM, Chen Song wrote:
> I got the same exception after the streaming job runs for a while, The
> ERROR message was complaining about a temp file no
I got the same exception after the streaming job runs for a while, The
ERROR message was complaining about a temp file not being found in the
output folder.
14/08/11 08:05:08 ERROR JobScheduler: Error running job streaming job
140774430 ms.0
java.io.FileNotFoundException: File
hdfs://hadoopc/u
I just saw another error after my job was run for 2 hours:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException):
No lease on /apps/data/vddil/real-time/checkpoint/temp: File does not
exist. Holder DFSClient_NONMAPREDUCE_327993456_13 does not have any
Can you give a stack trace and logs of the exception? Its hard to say
anything without any associated stack trace and logs.
TD
On Fri, Jul 25, 2014 at 1:32 PM, Bill Jay
wrote:
> Hi,
>
> I am running a Spark Streaming job that uses saveAsTextFiles to save
> results into hdfs files. However, it
Hi,
I am running a Spark Streaming job that uses saveAsTextFiles to save
results into hdfs files. However, it has an exception after 20 batches
result-140631234/_temporary/0/task_201407251119__m_03 does
not exist.
When the job is running, I do not change any file in the folder. Does
11 matches
Mail list logo
