Using FAIR mode.
If no other way. I think there is a limitation on number of parallel jobs
that spark can run. Is there a way that more number of jobs can run in
parallel. This is alright because, this sparkcontext would only be used
during web service calls.
I looked at spark configuration page
Are you using the scheduler in fair mode instead of fifo mode?
Sent from my iPhone
> On Sep 22, 2018, at 12:58 AM, Jatin Puri wrote:
>
> Hi.
>
> What tactics can I apply for such a scenario.
>
> I have a pipeline of 10 stages. Simple text processing. I train the data with
> the pipeline and
