If you don't send jobs to different pools, then they will all end up in the default pool. If you leave the intra-pool scheduling policy as the default FIFO, then this will effectively be the same thing as using the default FIFO scheduling.
Depending on what you are trying to accomplish, you need some combination of multiple pools and FAIR scheduling within one or more pools. And. of course, you need to actually place a job within an appropriate pool. On Fri, May 15, 2015 at 11:13 AM, Evo Eftimov <[email protected]> wrote: > No pools for the moment – for each of the apps using the straightforward > way with the spark conf param for scheduling = FAIR > > > > Spark is running in a Standalone Mode > > > > Are you saying that Configuring Pools is mandatory to get the FAIR > scheduling working – from the docs it seemed optional to me > > > > *From:* Tathagata Das [mailto:[email protected]] > *Sent:* Friday, May 15, 2015 6:45 PM > *To:* Evo Eftimov > *Cc:* user > *Subject:* Re: Spark Fair Scheduler for Spark Streaming - 1.2 and beyond > > > > How are you configuring the fair scheduler pools? > > > > On Fri, May 15, 2015 at 8:33 AM, Evo Eftimov <[email protected]> > wrote: > > I have run / submitted a few Spark Streaming apps configured with Fair > scheduling on Spark Streaming 1.2.0, however they still run in a FIFO mode. > Is FAIR scheduling supported at all for Spark Streaming apps and from what > release / version - e.g. 1.3.1 > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Fair-Scheduler-for-Spark-Streaming-1-2-and-beyond-tp22902.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > >
