NotEnoughReplicasException: The size of the current ISR Set(2) is insufficient to satisfy the min.isr requirement of 3

2020-07-07 Thread Nag Y
I had the following setup Brokers : 3 - all are up and running with min.insync.replicas=3. I created a topic with the following configuration bin\windows\kafka-topics --zookeeper 127.0.0.1:2181 --topic topic-ack-all --create --partitions 4 --replication-factor 3 I triggered the producer with "ac

Re: NotEnoughReplicasException: The size of the current ISR Set(2) is insufficient to satisfy the min.isr requirement of 3

2020-07-07 Thread Liam Clarke-Hutchinson
Hi Nag, ISR is the replicas that are in sync with the leader, and there's a different ISR set for each partition of a given topic. If you use `kafka/bin/kafka-topics --describe --topic ` it'll show you the replicas and ISR for each partition. min.insync.replicas and replication factor are all abo

how to make a user as consumer to all topics using kafka-acls.sh

2020-07-07 Thread 张祥
Hi community, I need to grant a user consumer permissions on all topics (super consumer ), including new topics that haven’t been created. I have tried this: kafka-acls.sh --add --allow-principal User:$1 --operation Read --topic "*" kafka-acls.sh --add --allow-principal User:$1 --operation Descri

Re: Keys and partitions

2020-07-07 Thread Vinicius Scheidegger
Hi Victoria, If processing order is not a requirement you could define a random key and your load would be randomly distributed across partitions. So far I was unable to find a solution to perfectly distribute the load across partitions when records are created from multiple producers - random dis

Re: Keys and partitions

2020-07-07 Thread Ricardo Ferreira
It is also important to note that since the release 2.4 of Apache Kafka the DefaultPartitioner now implements a sticky partitioning strategy rather than round-robin based on the key. This means that if you need fine control over which partition records will end up given the key -- you ought to

Re: destination topics in mm2 larger than source topic

2020-07-07 Thread Ricardo Ferreira
Iftach, This is a very useful finding. While I don't know the answer to your question below, I would like to take this opportunity to encourage you to write a blog about this finding =) Thanks, -- Ricardo On 7/7/20 2:48 AM, Iftach Ben-Yosef wrote: I believe I got it to work with "source->d

Re: Problem with replication?!

2020-07-07 Thread Ricardo Ferreira
Given the stack trace you've shared below I can tell that this *is not a replication issue* but rather -- your producer is not being able to write records into the partitions because the brokers that host them are unavailable. Now, I know that they are indeed running so "unavailable" here means