Re: ACL with SSL is not working

2017-05-19 Thread Michael Rauter
Hi, with SSL client authentication the user identifier is the dname of the certificate in your case “CN=Bob,O=FB,OU=MA,L=MP,ST=CA,C=US” for example when you want to set an ACL rule (read and write for topic TOPICNAME from every host): $ kafka-acls --authorizer-properties zookeeper.connect=zo

Re: Kafka Upgrade 0.10.2.0 to 0.10.2.1 partitionRecordsOrFail

2017-05-19 Thread Michael Rauter
Thanks (also to Guozhang), I also realized that change, that should be the cause.. Am 19.05.17, 19:20 schrieb "isma...@gmail.com im Auftrag von Ismael Juma" : Your classpath seems to contain more than one version of the kafka-clients jars (probably 0.10.2.0 and 0.10.2.1). partitionRec

Re: ACL with SSL is not working

2017-05-19 Thread Raghav
If it helps, this is how I generated the keystone for my client $ keytool -alias kafka-dev2 -validity 365 -keystore kafka-dev2.keystore.jks -dname "CN=Bob,O=FB,OU=MA,L=MP,ST=CA,C=US" -genkey -ext SAN=DNS: kafka-dev2.example.com -storepass 123456 Anything wrong here ? On Fri, May 19, 2017 at 10:3

Re: Kafka Upgrade 0.10.2.0 to 0.10.2.1 partitionRecordsOrFail

2017-05-19 Thread Guozhang Wang
Hi Michael, Could you double check if your old jar has been cleaned up on the location that the new jar is copied to? From the stack trace "partitionRecordsOrFail" it seems the old jar is still loaded. Guozhang On Fri, May 19, 2017 at 7:54 AM, Michael Rauter wrote: > I try to do a rolling upg

ACL with SSL is not working

2017-05-19 Thread Raghav
Hi I have a SSL setup with Kafka Broker, Producer and Consumer, and it works fine. I tried to setup ACLs as given on the website. When I start my producer, I am getting this error: [root@kafka-dev2 KAFKA]# bin/kafka-console-producer --broker-list kafka-dev1.example.com:9093 --topic test --produc

Re: Kafka Upgrade 0.10.2.0 to 0.10.2.1 partitionRecordsOrFail

2017-05-19 Thread Ismael Juma
Your classpath seems to contain more than one version of the kafka-clients jars (probably 0.10.2.0 and 0.10.2.1). partitionRecordsOrFail was added in 0.10.2.1 and the class from the older jar was loaded instead. Can you give a little more details regarding the Kafka package you are using and how yo

Re: Kafka ACL's with SSL Protocol is not working

2017-05-19 Thread Raghav
Hi I have very similar problem that Raghu faced. How can I enable the log4j debug mode ? Thanks. On Thu, Dec 15, 2016 at 3:32 PM, Raghu B wrote: > Thanks Derar & Kiran, your suggestions are very useful. > > I enabled Log4J debug mode and found that my client is trying to connect to > the Kafk

Re: Data loss after a Kafka Broker restart scenario

2017-05-19 Thread Jun Rao
Hi, Fathima, Did you check if produced messages are acked successfully? Only successfully acked messages are guaranteed to be preserved on the broker. Thanks, Jun On Thu, May 18, 2017 at 12:10 AM, Fathima Amara wrote: > Hi Jun, > > Thanks alot for the reply. As suggested, I tried running my a

Re: Increase partion in 10.2, unknownpartition error

2017-05-19 Thread Naanu Bora
And it seems that the partition files are missing in leader broker..is there any way I can recover this topic since delete topic seems to have issues ? On May 19, 2017 8:40 AM, "Naanu Bora" wrote: > Hi, > I am using kafka manager to increase the partion of a topic. After > increasing the

Kafka Upgrade 0.10.2.0 to 0.10.2.1 partitionRecordsOrFail

2017-05-19 Thread Michael Rauter
I try to do a rolling upgrade a kafka cluster from 0.10.2.0 to 0.10.2.1. But after upgrade the first node to 0.10.2.1 I get: [2017-05-19 16:02:03,941] ERROR [KafkaApi-3] Error when handling request {acks=-1,timeout=3,topic_data=[{topic=topic data=[{partition=4,record_set=[(offset=0,record=R

Re: Messages are repeating in kafka

2017-05-19 Thread Girish Aher
Hi Abhimanyu, What is your offsets.retention.minutes on brokers set to? It could have happened that the offsets in __consumer_offsets topic got wiped out after the configured time and so when the consumer restarted, it did not find where to start with. If auto.offset.reset is set to earliest on co

Increase partion in 10.2, unknownpartition error

2017-05-19 Thread Naanu Bora
Hi, I am using kafka manager to increase the partion of a topic. After increasing the number of partitions I am seeing following error for one partition and writes to this partition hangs. Logs in leader of partition 3: While recording the replica LEO, the partition testtopic-2 hasn't been c

Messages are repeating in kafka

2017-05-19 Thread Abhimanyu Nagrath
I am using single node Kafka broker (v 0.10.2) Hardware (8 core, 16 GB RAM, 1 TD HardDisk) and zookeeper (v 3.4.8). I have a topic with 200 partition in which messages contains the total of 3 Million messages. It took 5 days to completely process all the messages and as soon as message got processe