Re: Flink kafka consumers stopped consuming messages

2021-06-09 Thread Ilya Karpov
Hi Arvid, thanks for reply, thread dump + logs research didn’t help. We suggested that problem was in async call to remote key-value storage because we (1) found that async client timeout was set to 0 (effectively no timeout, idle infinitely), (2) async client threads we sleeping, (3) AsyncWait

Re: Flink kafka consumers stopped consuming messages

2021-06-09 Thread Arvid Heise
Hi Ilya, These messages could pop up when a Kafka broker is down but should eventually disappear. So I'm a bit lost. If there was a bug, it's also most likely fixed in the meantime. So if you want to be on the safe side, I'd try to upgrade to more recent versions (Flink + Kafka consumer). Best,

Flink kafka consumers stopped consuming messages

2021-06-02 Thread Ilya Karpov
Hi there, today I've observed strange behaviour of a flink streaming application (flink 1.6.1, per-job cluster deployment, yarn): 3 task managers (2 slots each) are running but only 1 slot is actually consuming messages from kafka (v0.11.0.2), others were idling (currentOutputWatermark was stuc