Re: Allowing parallelism in spark local mode

2016-02-12 Thread Chris Fregly
.@granturing.com> wrote: > You’ll want to setup the FAIR scheduler as described here: > https://spark.apache.org/docs/latest/job-scheduling.html#scheduling-within-an-application > > From: yael aharon > Date: Friday, February 12, 2016 at 2:00 PM > To: "user@spark.apache.org

Re: Allowing parallelism in spark local mode

2016-02-12 Thread Silvio Fiorito
lt;mailto:user@spark.apache.org>" mailto:user@spark.apache.org>> Subject: Allowing parallelism in spark local mode Hello, I have an application that receives requests over HTTP and uses spark in local mode to process the requests. Each request is running in its own thread. It seems that spark is q

Allowing parallelism in spark local mode

2016-02-12 Thread yael aharon
Hello, I have an application that receives requests over HTTP and uses spark in local mode to process the requests. Each request is running in its own thread. It seems that spark is queueing the jobs, processing them one at a time. When 2 requests arrive simultaneously, the processing time for each