Subject: Re: Issues with partitionBy: FetchFailed
Hi Jerry,
For the one executor hung with one CPU core running at 100%, that sounds
exactly like the symptoms I observed in
https://issues.apache.org/jira/browse/SPARK-2546 around a deadlock with the
JobConf. The next time that happens can you
bably some potential bugs may introduced.
>
>
>
> Thanks
>
> Jerry
>
>
>
> *From:* David Rowe [mailto:davidr...@gmail.com]
> *Sent:* Monday, September 22, 2014 7:12 PM
> *To:* Andrew Ash
> *Cc:* Shao, Saisai; Julien Carme; user@spark.apache.org
> *Subject:* Re: Iss
y be different from the previous code,
> I guess probably some potential bugs may introduced.
>
>
>
> Thanks
>
> Jerry
>
>
>
> *From:* David Rowe [mailto:davidr...@gmail.com]
> *Sent:* Monday, September 22, 2014 7:12 PM
> *To:* Andrew Ash
> *Cc:* Shao, Saisai;
from the previous code, I guess
probably some potential bugs may introduced.
Thanks
Jerry
From: David Rowe [mailto:davidr...@gmail.com]
Sent: Monday, September 22, 2014 7:12 PM
To: Andrew Ash
Cc: Shao, Saisai; Julien Carme; user@spark.apache.org
Subject: Re: Issues with partitionBy: FetchFailed
timeout, default is 60 seconds.
>>>
>>>
>>>
>>> Sometimes because of GC pause or some other reasons, acknowledged
>>> message will be timeout, which will lead to this exception, you can try
>>> setting a large value of this configuration.
>>
ons, acknowledged message
>> will be timeout, which will lead to this exception, you can try setting a
>> large value of this configuration.
>>
>>
>>
>> Thanks
>>
>> Jerry
>>
>>
>>
>> *From:* Julien Carme [mailto:julien.ca...
gt;
>
> *From:* Julien Carme [mailto:julien.ca...@gmail.com]
> *Sent:* Sunday, September 21, 2014 7:43 PM
> *To:* user@spark.apache.org
> *Subject:* Issues with partitionBy: FetchFailed
>
>
>
> Hello,
>
> I am facing an issue with partitionBy, it is not clear whether it is a
&
, you can try setting a large
value of this configuration.
Thanks
Jerry
From: Julien Carme [mailto:julien.ca...@gmail.com]
Sent: Sunday, September 21, 2014 7:43 PM
To: user@spark.apache.org
Subject: Issues with partitionBy: FetchFailed
Hello,
I am facing an issue with partitionBy, it is not
Hello,
I am facing an issue with partitionBy, it is not clear whether it is a
problem with my code or with my spark setup. I am using Spark 1.1,
standalone, and my other spark projects work fine.
So I have to repartition a relatively large file (about 70 million lines).
Here is a minimal version
