Re: executorAdded event to DAGScheduler

2014-09-26 Thread praveen seluka
not sure if it will create an issue when you have multiple workers in > the same host, as submitWaitingStages is called everywhere and I never > try such a deployment mode > > Best, > > -- > Nan Zhu > > On Friday, September 26, 2014 at 8:02 AM, praveen seluka wrote: > &

Re: executorAdded event to DAGScheduler

2014-09-26 Thread praveen seluka
Some corrections. On Fri, Sep 26, 2014 at 5:32 PM, praveen seluka wrote: > Can someone explain the motivation behind passing executorAdded event to > DAGScheduler ? *DAGScheduler *does *submitWaitingStages *when *executorAdded > *method is called by *TaskSchedulerImpl*. I see some iss

executorAdded event to DAGScheduler

2014-09-26 Thread praveen seluka
Can someone explain the motivation behind passing executorAdded event to DAGScheduler ? *DAGScheduler *does *submitWaitingStages *when *executorAdded *method is called by *TaskSchedulerImpl*. I see some issue in the below code, *TaskSchedulerImpl.scala code* if (!executorsByHost.contains(o.host))