RE: Frequent Consumer Rebalance/ Commit fail exception

2016-10-11 Thread Misra, Rahul
onveniently supported than pause-and-poll pattern. Hope it helps. Guozhang On Mon, Oct 10, 2016 at 12:13 PM, Misra, Rahul wrote: > Hi, > > I have a custom Kafka consumer which reads messages from a topic, > hands over the processing of the messages to a different thread, and

Frequent Consumer Rebalance/ Commit fail exception

2016-10-10 Thread Misra, Rahul
Hi, I have a custom Kafka consumer which reads messages from a topic, hands over the processing of the messages to a different thread, and while the messages are being processed, it pauses the topic and keeps polling the Kafka topic (to maintain heartbeats) and also commits offsets using commi

Same leader for all partitions for topics

2016-10-07 Thread Misra, Rahul
Hi, I have been using a 3 node kafka cluster for development for some time now. I have created some topics on this cluster. Yesterday I observed the following when I used 'describe' for the topics: The Kafka version I'm using is: 9.0.1 (kafka_2.11-0.9.0.1). Topic:topicIc PartitionCount:3

RE: Offsets getting lost if no messages sent for a long time

2016-08-22 Thread Misra, Rahul
Misra -----Original Message- From: Misra, Rahul [mailto:rahul.mi...@altisource.com] Sent: Sunday, August 21, 2016 12:46 AM To: Ian Wrigley; users@kafka.apache.org Subject: RE: Offsets getting lost if no messages sent for a long time Hi Ian, Thanks for the quick response. Your answer clears

RE: Offsets getting lost if no messages sent for a long time

2016-08-20 Thread Misra, Rahul
nt, Inc > On Aug 20, 2016, at 11:36 AM, Misra, Rahul wrote: > > Hi, > > I have observed the following scenario (the consumer here has > 'enable.auto.commit=false' and offsets are committed using commitSync() if > any messages are received): > > 1. St

Offsets getting lost if no messages sent for a long time

2016-08-20 Thread Misra, Rahul
Hi, I have observed the following scenario (the consumer here has 'enable.auto.commit=false' and offsets are committed using commitSync() if any messages are received): 1. Start a consumer (with a specific group.Id) and send some messages to its subscribed topic. 2. The consumer con

RE: Kafka broker crash

2016-06-22 Thread Misra, Rahul
may delete the files. So it is throwing FileNotFoundException. you can change the file location to some other path and restart all broker. This might fix the issue. Regrads, Madhukar On Wed, Jun 22, 2016 at 1:40 PM, Misra, Rahul wrote: > Hi, > > I'm facing a strange issue in my Kafka cluster. Co

Kafka broker crash

2016-06-22 Thread Misra, Rahul
Hi, I'm facing a strange issue in my Kafka cluster. Could anybody please help me with it. The issue is as follows: We have a 3 node kafka cluster. We installed the zookeeper separately and have pointed the brokers to it. The zookeeper is also 3 node, but for our POC setup, the zookeeper nodes