How did you configure YARN queues? What scheduler? Preemption ? > On 19 Feb 2016, at 06:51, Prabhu Joseph <prabhujose.ga...@gmail.com> wrote: > > Hi All, > > When running concurrent Spark Jobs on YARN (Spark-1.5.2) which share a > single Spark Context, the jobs take more time to complete comparing with when > they ran with different Spark Context. > The spark jobs are submitted on different threads. > > Test Case: > > A. 3 spark jobs submitted serially > B. 3 spark jobs submitted concurrently and with different SparkContext > C. 3 spark jobs submitted concurrently and with same Spark Context > D. 3 spark jobs submitted concurrently and with same Spark Context and > tripling the resources. > > A and B takes equal time, But C and D are taking 2-3 times longer than A, > which shows concurrency does not improve with shared Spark Context. [Spark > Job Server] > > Thanks, > Prabhu Joseph
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
