Hello ! I contact you because I'm have a problem with my architecture using kafka.
I'm using Confluent 1.0 with kafka 0.8.2. I have the following architecture : - A job who calls every hour the kafka REST API behind a load-balancer. - The Kafka REST API servers points on a kafka cluster of three brokers. - The zookeeper servers are located on kafka brokers. This problem concerns one topic with only one partition. It is consumed by only one consumer group. The process is the following : 0. There is a process running very often which produces new messages in the topic. 1. Each hour, the job create a consumer instance for the consumer group (POST request) through the load balancer. Then it gets the REST node where the consumer instance was created. 2. Then, the job consume the messages through a GET request with this consumer instance on the right REST node. 3. Once it is done, it commits the offset 4. And finally, once everything done, it deletes the consumer instance I noticed the following problem : One hour the job will work and get new messages. And then during several hours, the command to retrieve the new messages will get nothing new, even if there is new messages in the topic, and even if the offset of the last consumed message is not the last. Technically speaking, the REST query returns a 200 code with a body empty, when the problem occured. I checked in zookeeper, all three zookeepers are well synchronized, and the offset is the same on all three zookeeper servers for this consumergroup for this topic. I checked in kafka and it says the replicas are well insynced. I was wonderinng what could explain this behaviour ? I checked the kafka logs and see nothing which could explain the errors unfortunately. Any idea where I could look to find the explanations of this behaviour ? Best regards. M.
