Hi, In the current flume implementation, the sink runner implements a backoff mechanism that kicks in when there is not enough data is coming in (ie it tries to slow down the sink).
The backoff starts off at 1s sleep and increments at 1s interval until reaching maximum 5s sleep time. These backoff settings (1s, 5s) are hardcoded and can’t be adjusted. This presents a real challenge when dealing with data that is coming say every few seconds yet it is desired that flume not introduce any latency. Also, in this case, slowing down the sink is not a desired goal. Is there any reason for not exposing the “backoff settings” through flume’s configuration? Thanks, Karim
