Kafka Behavior During Partition Leader Shutdowns

Hello, I've been working on testing Kafka availability in Zookeeper mode during single broker shutdowns within a Kubernetes setup, and I've come across something interesting that I wanted to run by you. We've noticed that when a partition leader goes down, messages are not delivered until a

A Least Effort Kafka cluster rebalancer

Hi all, I created a Kafka rebalancer (in bash) which minimizes the number of partitions to be moved around when a broker (or more than one) is added or removed from the cluster. Contrarily to the already available 'kafka-reassign-partitions' (which comes with Kafka package), this new tool work

How does kafka consumer behave when consumer poll timeout has expired?

Hi Can someone let me know how a consumer is expected to behave after the below log? Will the consumer be considered dead and a new instance will be spawned due to consumer group rebalancing? How is this behaviour with RangeAssignor and CooperativeStickyAssginer? consumer poll timeout has expired

Re: Kafka Streams reaching ERROR state during rolling upgrade / restart of brokers

Matthias It happened again yesterday during another rolling update. The first error log I can see on the client side is below. It was there in PENDING_ERROR state for sometime and then went into ERROR state. Caused by: java.lang.IllegalStateException: KafkaStreams is not running. State is PENDING