Yes that's exactly what happens, but I would think that if data node is
unavailable/unavailability of data for one of the nodes should not cause
indefinite wait.. Are there any properties we can set to avoid getting into
indefinite/non-deterministic outcome of a spark application?
On Thu, May 16,
One of the reason that any jobs running on YARN (Spark, MR, Hive, etc) can
get stuck is if there is data unavailability issue with HDFS.
This can arise if either the Namenode is not reachable or if the particular
data block is unavailable due to node failures.
Can you check if your YARN service ca
on yarn
On Thu, May 16, 2019 at 1:36 AM Akshay Bhardwaj <
akshay.bhardwaj1...@gmail.com> wrote:
> Hi Rishi,
>
> Are you running spark on YARN or spark's master-slave cluster?
>
> Akshay Bhardwaj
> +91-97111-33849
>
>
> On Thu, May 16, 2019 at 7:15 AM Rishi Shah
> wrote:
>
>> Any one please?
>>
>
Hi Rishi,
Are you running spark on YARN or spark's master-slave cluster?
Akshay Bhardwaj
+91-97111-33849
On Thu, May 16, 2019 at 7:15 AM Rishi Shah wrote:
> Any one please?
>
> On Tue, May 14, 2019 at 11:51 PM Rishi Shah
> wrote:
>
>> Hi All,
>>
>> At times when there's a data node failure,
Any one please?
On Tue, May 14, 2019 at 11:51 PM Rishi Shah
wrote:
> Hi All,
>
> At times when there's a data node failure, running spark job doesn't fail
> - it gets stuck and doesn't return. Any setting can help here? I would
> ideally like to get the job terminated or executors running on tho
