Drop Topic from Consumer

We are using Kafka 0.8.1 and have a Consumer that used to pull multiple topics. One of these topics is no longer needed by this consumer/group, so when I look at /usr/local/kafka/bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --zkconnect localhost --group We see the LAG for this meas

How Kafka Distribute partitions across the cluster

Hi All, I would like to know what kind of strategy Kafka adopts in splitting partitions of a topic to across the cluster. Ex: if i have 10 broker in a cluster. 1 topic with 5 partition and each with replication-factor of 2. then how would Kafka will assign the partitions ( 5*2 =10) across the clu

Re: How Kafka Distribute partitions across the cluster

Hi Nitin, Partitions and replicas are assigned in a round robin fashion. You can read more about it in the docs under "4.7 Replication " look for the header for "Replica Management". You can view the implementation here: https://github.com/a

Re: Drop Topic from Consumer

Is it a wildcard consumer or you specify the topics manually? You can do this by modifying the znode in zookeeper for that consumer group. Thanks, Mayuresh On Tue, Oct 27, 2015 at 8:09 AM, Casey Daniell wrote: > We are using Kafka 0.8.1 and have a Consumer that used to pull multiple > topics.

Re: How Kafka Distribute partitions across the cluster

Thanks Grant, This is really helpful.. Regards, Nitin Kumar Sharma. On Tue, Oct 27, 2015 at 1:16 PM, Grant Henke wrote: > Hi Nitin, > > Partitions and replicas are assigned in a round robin fashion. You can read > more about it in the docs under "4.7 Replication >

Kafka and Btrfs

Anyone using Kafka with Brtfs successfully? Any recommendations against taking that path? Elias

Produce request failed due to NotLeaderForPartitionException (leader epoch is old)

Hi Ivan, I face the same issue you had. Were you able to figure out why it’s happening? If so, could you please share it. Thanks, Chaitanya GSK

Re: Questions about .9 consumer API

The Java docs have not been updated, but here is a short summary: Consumer.poll() call will possibly throw three types of exceptions: 1) WakeupException: this is when Consumer.close() is called by another thread which interrupts another user thread polling on the consumer, users should really cat

Occasional Time Outs - GC problem or...?

Just poking around our amazon kafka instance. Occasionally one of our applications gets a kafka time-out. We want to guess it is a momentary network issue with aws, however, I noticed the GC log is updated very frequently. Could the GC be causing a problem? How do I determine if it is causing a p