Hi, What version of Kafka are you using?
Ismael On Fri, Feb 15, 2019 at 8:32 PM Ankur Rana <ankur.r...@getfareye.com> wrote: > Any comments anyone? > > On Fri, Feb 15, 2019 at 6:08 PM Ankur Rana <ankur.r...@getfareye.com> > wrote: > >> Hi everyone, >> >> We have a Kafka cluster with 5 brokers with all topics having at least 2 >> replication factor. We have multiple Kafka consumers applications running >> on this cluster. Most of these consumers are build using consumer APIs and >> quite recently we have started using Stream applications. >> >> We are facing a really weird issue. Just Sometimes it happens that our >> Kafka cluster breaks down, By breaking down I mean that consumers and >> producers start throwing disconnection exception and all of them just stop. >> >> We use debezium connector to push Postgres events to Kafka topics. >> Debezium throws the error below: >> [image: image.png] >> >> >> Kafka broker throws the error below: >> COORDINATOR_NOT_AVAILABLE >> [image: image.png] >> >> >> Error on the consumer side : >> >> [image: image.png] >> >> >> In order to fix, I stop the disconnected broker and everything fixes >> itself. Debezium starts flushing messages and all consumers start working >> normally. I bring the disconnected broker up and everything works as >> before without any problem. >> >> I don't understand a few things here : >> >> >> 1. what could be the reason behind this disconnection exception. Even >> if one of the broker was somehow disconnected, Isn't kafka suppose to >> handle it in a cluster where all topics have a replication factor of 2. >> 2. It appears that the malfunctioning broker was in a state where it >> was neither disconnected nor connected to the cluster. I could still see >> the broker visible in Kafka manager with zero bytes In, while it was >> disconnected from all the producers and consumers. >> 3. Weirdly, I have noticed that this situation usually occurs when I >> start the multiple consumers of the stream application. Not sure about >> this >> as this error has only occurred a few times. It happened twice today and >> both the times I started 3 consumers of the same stream application. >> >> >> Can anyone help me debug this problem. I don't know where to look for >> possible issues with our cluster or stream application. I am attaching >> streams config and stream application code for your reference. >> Please feel free to ask for any more details. >> >> >> Stream config : >> [image: image.png] >> >> >> Stream application code : https://codeshare.io/Gq6pLB >> >> -- >> Thanks, >> >> Ankur Rana >> Software Developer >> FarEye >> > > > -- > Thanks, > > Ankur Rana > Software Developer > FarEye >
