Re: Spark Fair Scheduler for Spark Streaming - 1.2 and beyond

2015-05-16 Thread Tathagata Das
pported for Spark Streaming Apps >> >> >> >> *From:* Richard Marscher [mailto:rmarsc...@localytics.com] >> *Sent:* Friday, May 15, 2015 7:20 PM >> *To:* Evo Eftimov >> *Cc:* Tathagata Das; user >> *Subject:* Re: Spark Fair Scheduler for Spark Streaming - 1

Re: Spark Fair Scheduler for Spark Streaming - 1.2 and beyond

2015-05-15 Thread Mark Hamstra
Tathagata Das [mailto:t...@databricks.com] > *Sent:* Friday, May 15, 2015 6:45 PM > *To:* Evo Eftimov > *Cc:* user > *Subject:* Re: Spark Fair Scheduler for Spark Streaming - 1.2 and beyond > > > > How are you configuring the fair scheduler pools? > > > > On Fri,

Re: Spark Fair Scheduler for Spark Streaming - 1.2 and beyond

2015-05-15 Thread Richard Marscher
mation that FAIR scheduling is > supported for Spark Streaming Apps > > > > *From:* Richard Marscher [mailto:rmarsc...@localytics.com] > *Sent:* Friday, May 15, 2015 7:20 PM > *To:* Evo Eftimov > *Cc:* Tathagata Das; user > *Subject:* Re: Spark Fair Scheduler for Spark St

RE: Spark Fair Scheduler for Spark Streaming - 1.2 and beyond

2015-05-15 Thread Evo Eftimov
Cc: Tathagata Das; user Subject: Re: Spark Fair Scheduler for Spark Streaming - 1.2 and beyond The doc is a bit confusing IMO, but at least for my application I had to use a fair pool configuration to get my stages to be scheduled with FAIR. On Fri, May 15, 2015 at 2:13 PM, Evo Eftimov

Re: Spark Fair Scheduler for Spark Streaming - 1.2 and beyond

2015-05-15 Thread Richard Marscher
t...@databricks.com] > *Sent:* Friday, May 15, 2015 6:45 PM > *To:* Evo Eftimov > *Cc:* user > *Subject:* Re: Spark Fair Scheduler for Spark Streaming - 1.2 and beyond > > > > How are you configuring the fair scheduler pools? > > > > On Fri, May 15, 2015 at 8:3

RE: Spark Fair Scheduler for Spark Streaming - 1.2 and beyond

2015-05-15 Thread Evo Eftimov
me From: Tathagata Das [mailto:t...@databricks.com] Sent: Friday, May 15, 2015 6:45 PM To: Evo Eftimov Cc: user Subject: Re: Spark Fair Scheduler for Spark Streaming - 1.2 and beyond How are you configuring the fair scheduler pools? On Fri, May 15, 2015 at 8:33 AM, Evo Eftimov wrote

Re: Spark Fair Scheduler for Spark Streaming - 1.2 and beyond

2015-05-15 Thread Tathagata Das
How are you configuring the fair scheduler pools? On Fri, May 15, 2015 at 8:33 AM, Evo Eftimov wrote: > I have run / submitted a few Spark Streaming apps configured with Fair > scheduling on Spark Streaming 1.2.0, however they still run in a FIFO mode. > Is FAIR scheduling supported at all for S