spark jobs unstable.
Thx
tri
From: Veljko Skarich [mailto:veljko.skar...@gmail.com]
Sent: Tuesday, December 15, 2015 3:08 PM
To: user@spark.apache.org
Subject: ideal number of executors per machine
Hi,
I'm looking for suggestions on the ideal number of executors per machine. I run
my jobs o
machine is the right number. If you are running very large heaps
>> (>64GB) you may consider multiple per machine just to make sure each's
>> GC pauses aren't excessive, but even this might be better mitigated
>> with GC tuning.
>>
>> On Tue, Dec 15, 2015 at
> Hi,
>
> I'm looking for suggestions on the ideal number of executors per machine. I
> run my jobs on 64G 32 core machines, and at the moment I have one executor
> running per machine, on the spark standalone cluster.
>
> I could not find many guidelines for figuring o
;
> On 15 December 2015 at 13:07, Veljko Skarich <mailto:veljko.skar...@gmail.com>> wrote:
> Hi,
>
> I'm looking for suggestions on the ideal number of executors per machine. I
> run my jobs on 64G 32 core machines, and at the moment I have one executor
> running
process
switches and therefore having a single executor with multiple threads gives
a better over-all performance that multiple executors with less threads.
--Jakob
On 15 December 2015 at 13:07, Veljko Skarich
wrote:
> Hi,
>
> I'm looking for suggestions on the ideal number of executo
Hi,
I'm looking for suggestions on the ideal number of executors per machine. I
run my jobs on 64G 32 core machines, and at the moment I have one executor
running per machine, on the spark standalone cluster.
I could not find many guidelines for figuring out the ideal number of
executors
