Re: Kafka Streams TimeoutException and keeps rebalancing

2019-10-24 Thread Xiyuan Hu
Thanks Matthias, I think it's related to https://issues.apache.org/jira/browse/KAFKA-8802. Once I disabled the cache, bytes out goes up a lot. Before the bytes out are kept at 14MB/S no matter bytes in is high or low. I'm trying to download the 2.3.1-rc2 build and try it again. On Thu, Oct 24, 2

Re: Kafka Streams TimeoutException and keeps rebalancing

2019-10-23 Thread Matthias J. Sax
>> 1) The app has a huge consuming lag on both source topic and internal >> repartition topics , ~5 M messages and keeps growing. Will the lag >> lead to this timeout exception? My understanding is the app polls too >> many messages before it could send out even the lag indicates it still >> polls

Kafka Streams TimeoutException and keeps rebalancing

2019-09-27 Thread Xiyuan Hu
Hi, I'm running Kafka Streams v2.1.0 with windowing function and 3 threads per node. The input traffic is about 120K messages/sec. Once deploy, after couple minutes, some thread will get TimeoutException and goes to DEAD state. 2019-09-27 13:04:34,449 ERROR [client-StreamThread-2] o.a.k.s.p.i.Ass