Ah, nevermind, I don't know anything about scheduling tasks in YARN. On Thu, Aug 13, 2015 at 11:03 PM, Ara Vartanian <[email protected]> wrote:
> I’m running on Yarn. > > On Aug 13, 2015, at 10:58 PM, Philip Weaver <[email protected]> > wrote: > > Are you running on mesos, yarn or standalone? If you're on mesos, are you > using coarse grain or fine grained mode? > > On Thu, Aug 13, 2015 at 10:13 PM, Ara Vartanian <[email protected]> > wrote: > >> I’m observing an unusual situation where my step duration increases as I >> add further executors to my cluster. My algorithm is fully data >> parallelizable into a map phase, followed by a reduce step at the end that >> amounts to matrix addition. So I’ve kicked a cluster of, say, 100 executors >> with 4 cores per executor and before running the algorithm I’ve >> repartitioned the RDD into 400 partitions. I can see in the Spark UI that >> each of the 400 (map) tasks takes about 2 seconds. However, the entire step >> is taking over a minute, and this is because the launch times of the tasks >> as reported in the Spark UI are staggered. For example, the first 100 might >> be launched in the same second, then another group 3 seconds later, and so >> forth (with the durations slowly expanding). With a task time of 2 seconds, >> this “launch lag” is dominating the computation time and only gets worse as >> I add nodes. >> >> Any insight on how I could go about diagnosing this would be greatly >> appreciated. >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> > >
