Hello, I wanted to follow up to see if anyone has ever experienced such behavior with Kafka Connect? Any guidance would be greatly appreciated!
Thanks, Upesh Upesh Desai | Senior Software Developer | ude...@itrsgroup.com www.itrsgroup.com From: Upesh Desai <ude...@itrsgroup.com> Date: Thursday, April 22, 2021 at 4:10 PM To: Users <users@kafka.apache.org> Subject: Kafka Connect Dist. Worker Does not join group Hello, Has anyone experienced this scenario before when having a distributed connect cluster: [kafka-coordinator-heartbeat-thread | connect-cluster] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Worker clientId=connect-1, groupId=connect-cluster] Member connect-1-50ee335c-6a0c-42d6-b3fc-0dc43067de0b sending LeaveGroup request to coordinator kafka-1:9092 (id: 2147483646 rack: null) due to consumer poll timeout has expired. This mean s the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time processing messages. You can address this either by increasing max.poll.interval.ms or by reducing the maximum size of batches returned in poll() with max.poll.records. I have left max.poll.interval.ms at the default of 5 minutes, and changed rebalance.timeout.ms=300000, yet still see this error occurring. This was with 2 connect worker instances, deployed on Kubernetes, and we see this error on a fresh install (no previous kafka data), and also see it when re-deploying SOME of the time, it is not consistent behavior. Is there some other configuration I am missing? Thanks in advance, Upesh [cid:image001.png@01D740DE.44D9A2B0]<https://www.itrsgroup.com/> Upesh Desai Senior Software Developer ude...@itrsgroup.com<mailto:ude...@itrsgroup.com> www.itrsgroup.com<https://www.itrsgroup.com/> Internet communications are not secure and therefore the ITRS Group does not accept legal responsibility for the contents of this message. Any view or opinions presented are solely those of the author and do not necessarily represent those of the ITRS Group unless otherwise specifically stated. [itrs.email.signature] Disclaimer The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful. This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.
