[Spark][Scheduler] Spark DAGScheduler scheduling performance hindered on JobSubmitted Event

DAGScheduler becomes a bottleneck in cluster when multiple JobSubmitted events has to be processed as DAGSchedulerEventProcessLoop is single threaded and it will block other tasks in queue like TaskCompletion. The JobSubmitted event is time consuming depending on the nature of the job (Example:

Re: Welcoming some new committers

Congrats all! Well deserved. On Sat, Mar 3, 2018 at 4:10 AM, Marco Gaido wrote: > Congratulations to you all! > > On 3 Mar 2018 8:30 a.m., "Liang-Chi Hsieh" wrote: > >> >> Congrats to everyone! >> >> >> Kazuaki Ishizaki wrote >> > Congratulations to everyone! >> > >> > Kazuaki Ishizaki >> > >>