Re: spark job scheduling

2016-01-27 Thread Jakob Odersky
like to understand more about >>> the scheduling system too. Scheduling strategy like FAIR or FIFO do have >>> significant impact on a Spark cluster architecture design decision. >>> >>> Best, >>> >>> Chayapan (A) >>> >>> On

Re: spark job scheduling

2016-01-27 Thread Chayapan Khannabha
. Scheduling strategy like FAIR or FIFO do have >> significant impact on a Spark cluster architecture design decision. >> >> Best, >> >> Chayapan (A) >> >> On Thu, Jan 28, 2016 at 10:07 AM, Niranda Perera < >> niranda.per...@gmail.com> wrote: >

Re: spark job scheduling

2016-01-27 Thread Niranda Perera
k cluster architecture design decision. > > Best, > > Chayapan (A) > > On Thu, Jan 28, 2016 at 10:07 AM, Niranda Perera > wrote: > >> hi all, >> >> I have a few questions on spark job scheduling. >> >> 1. As I understand, the smallest unit of w

Re: spark job scheduling

2016-01-27 Thread Chayapan Khannabha
sign decision. Best, Chayapan (A) On Thu, Jan 28, 2016 at 10:07 AM, Niranda Perera wrote: > hi all, > > I have a few questions on spark job scheduling. > > 1. As I understand, the smallest unit of work an executor can perform. In > the 'fair' scheduler mode, let's s

spark job scheduling

2016-01-27 Thread Niranda Perera
hi all, I have a few questions on spark job scheduling. 1. As I understand, the smallest unit of work an executor can perform. In the 'fair' scheduler mode, let's say a job is submitted to the spark ctx which has a considerable amount of work to do in a task. While such a '