Duplicate messages when consumers are restarted

2015-09-23 Thread Jananee S
We are using kafka 0.8.2.1 and are noticing a weird behavior when the high level consumers are restarted. We have two consumers (C1 and C2, processes on different machines) in a consumer group. The topic has two partitions. We use kafka as offsets storage and have auto commit disabled. Here is th

Re: Cascading failures on running out of disk space

2015-06-08 Thread Jananee S
#x27;ve seen too. > > Probably the most important thing to know, is that you don't want to let > your disks fill up, so if you can add early warning/monitoring so you can > take action before that happens, you'd avoid these scenarios with unclean > leader election,

Cascading failures on running out of disk space

2015-05-27 Thread Jananee S
Hi, We have the following setup - Number of brokers: 3 Number of zookeepers: 3 Default replication factor: 3 Offets Storage: kafka When one of our brokers ran out of disk space, we started seeing lot of errors in the broker logs at an alarming rate. This caused the other brokers also to run ou