Hi Askash
The event dropping problem also triggered by slow listener or large
number of events or both, the easy and simple way is change the config of
`spark.scheduler.listenerbus.eventqueue.capacity`, its default value is 10000.
But if after change the queue capacity to a more lager value, it still
happened, you should check more about listener.
> 在 2018年5月27日，15:45，Aakash Basu <[email protected]> 写道：
>
> Hi,
>
> I'm getting the below ERROR and WARN when running a little heavy calculation
> on a dataset -
>
> To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use
> setLogLevel(newLevel).
> 2018-05-27 12:51:11 ERROR AsyncEventQueue:70 - Dropping event from queue
> appStatus. This likely means one of the listeners is too slow and cannot keep
> up with the rate at which tasks are being started by the scheduler.
> 2018-05-27 12:51:11 WARN AsyncEventQueue:66 - Dropped
> com.codahale.metrics.Counter@1d8423d1 events from appStatus since Thu Jan 01
> 05:30:00 IST 1970.
> [Stage 7784:>(9 + 3) / 34][Stage 7786:(53 + 38) / 200][Stage 7789:(74 + 53) /
> 200]2018-05-27 12:52:14 WARN AsyncEventQueue:66 - Dropped
> com.codahale.metrics.Counter@1d8423d1 events from appStatus since Sun May 27
> 12:51:11 IST 2018.
> [Stage 7784:>(9 + 3) / 34][Stage 7786:(53 + 38) / 200][Stage 7789:(74 + 53) /
> 200]2018-05-27 12:53:14 WARN AsyncEventQueue:66 - Dropped
> com.codahale.metrics.Counter@1d8423d1 events from appStatus since Sun May 27
> 12:52:14 IST 2018.
> 2018-05-27 12:54:14 WARN AsyncEventQueue:66 - Dropped
> com.codahale.metrics.Counter@1d8423d1 events from appStatus since Sun May 27
> 12:53:14 IST 2018.
> [Stage 7784:>(9 + 3) / 34][Stage 7786:(53 + 38) / 200][Stage 7789:(74 + 53) /
> 200]2018-05-27 12:55:14 WARN AsyncEventQueue:66 - Dropped
> com.codahale.metrics.Counter@1d8423d1 events from appStatus since Sun May 27
> 12:54:14 IST 2018.
> [Stage 7784:>(9 + 3) / 34][Stage 7786:(53 + 38) / 200][Stage 7789:(74 + 53) /
> 200]2018-05-27 12:56:15 WARN AsyncEventQueue:66 - Dropped
> com.codahale.metrics.Counter@1d8423d1 events from appStatus since Sun May 27
> 12:55:14 IST 2018.
> [Stage 7784:>(9 + 3) / 34][Stage 7786:(53 + 38) / 200][Stage 7789:(74 + 53) /
> 200]2018-05-27 12:57:32 WARN AsyncEventQueue:66 - Dropped
> com.codahale.metrics.Counter@1d8423d1 events from appStatus since Sun May 27
> 12:56:15 IST 2018.
> [Stage 7784:>(9 + 3) / 34][Stage 7786:(53 + 38) / 200][Stage 7789:(74 + 53) /
> 200]2018-05-27 12:58:32 WARN AsyncEventQueue:66 - Dropped
> com.codahale.metrics.Counter@1d8423d1 events from appStatus since Sun May 27
> 12:57:32 IST 2018.
> [Stage 7784:>(9 + 3) / 34][Stage 7786:(53 + 38) / 200][Stage 7789:(74 + 53) /
> 200]2018-05-27 12:59:33 WARN AsyncEventQueue:66 - Dropped
> com.codahale.metrics.Counter@1d8423d1 events from appStatus since Sun May 27
> 12:58:32 IST 2018.
> [Stage 7784:>(9 + 3) / 34][Stage 7786:(53 + 38) / 200][Stage 7789:(74 + 53) /
> 200]2018-05-27 13:00:34 WARN AsyncEventQueue:66 - Dropped
> com.codahale.metrics.Counter@1d8423d1 events from appStatus since Sun May 27
> 12:59:33 IST 2018.
> [Stage 7784:>(9 + 3) / 34][Stage 7786:(53 + 38) / 200][Stage 7789:(74 + 53) /
> 200]2018-05-27 13:01:35 WARN AsyncEventQueue:66 - Dropped
> com.codahale.metrics.Counter@1d8423d1 events from appStatus since Sun May 27
> 13:00:34 IST 2018.
> [Stage 7784:>(9 + 3) / 34][Stage 7786:(53 + 38) / 200][Stage 7789:(74 + 53) /
> 200]2018-05-27 13:02:36 WARN AsyncEventQueue:66 - Dropped
> com.codahale.metrics.Counter@1d8423d1 events from appStatus since Sun May 27
> 13:01:35 IST 2018.
>
> Even though my job is not failing but why am I getting these?
>
> Thanks,
> Aakash.
