Hi Alex In fact I copied the same configuration that Rajini pasted above and it worked for me. You can try the same. Let me know if it doesn't work.
Thanks. On Fri, May 26, 2017 at 4:19 AM, Kamalov, Alex <alex.kama...@bnymellon.com> wrote: > Hey Raghav, > > > > Yes, I would very much love to get your configs, so I can model against it. > > > > Thanks again, > > > > Alex > > > > *From: *Raghav <raghavas...@gmail.com> > *Date: *Thursday, May 25, 2017 at 10:54 PM > *To: *Mike Marzo <precisionarchery...@gmail.com> > *Cc: *Darshan Purandare <purandare.dars...@gmail.com>, Rajini Sivaram < > rajinisiva...@gmail.com>, Users <users@kafka.apache.org>, Alex Kamalov < > alex.kama...@bnymellon.com> > *Subject: *Re: Kafka Authorization and ACLs Broken > > > > In SSL cert, there is a field which has a CN (Common Name). So when ACLs > are set, they are set for that CN. This is how the ACLs are configured and > matched against. I am still pretty new to Kafka in general, but this is how > I think it works. I can copy my config if you want. > > > > On Thu, May 25, 2017 at 12:51 PM, Mike Marzo < > precisionarchery...@gmail.com> wrote: > > Stupid question.... > > If u don't specify a jaas file how does the consumer and producer specify > the Id that acl's are configured against.... boy I am getting more and > more perplexed by this... > > mike marzo > 908 209-4484 <(908)%20209-4484> > > > > On May 24, 2017 9:29 PM, "Raghav" <raghavas...@gmail.com> wrote: > > Mike > > > > I am not using jaas file. I literally took the config Rajini gave in the > previous email and it worked for me. I am using ssl Kafka with ACLs. I am > not suing kerberos. > > > > Thanks. > > > > On Wed, May 24, 2017 at 11:29 AM, Mike Marzo < > precisionarchery...@gmail.com> wrote: > > I'm also having issues getting acls to work. Out of intereat, are you > starting ur brokers with a jaas file, if so do u mind sharing the client > and server side jaas entries so I can validate what I'm doing. > > mike marzo > 908 209-4484 > > On May 24, 2017 10:54 AM, "Raghav" <raghavas...@gmail.com> wrote: > > > Hi Rajini > > > > Thank you very much. It perfectly works. > > > > I think in my setup I was trying to use a CA (certificate authority) to > > sign the certificates from client and server, and then adding it to trust > > store and keystore. I think in that process, I may have messed > something. I > > will try above config with a CA to sign certificates. Hopefully that > would > > work too. > > > > Thanks a lot again. > > > > Raghav > > > > > > > > > > On Wed, May 24, 2017 at 7:04 AM, Rajini Sivaram <rajinisiva...@gmail.com > > > > wrote: > > > > > Raghav/Darshan, > > > > > > Can you try these steps on a clean installation of Kafka? It works for > > me, > > > so hopefully it will work for you. And then you can adapt to your > > scenario. > > > > > > *Create keystores and truststores:* > > > > > > keytool -genkey -alias kafka -keystore server.keystore.jks -dname > > > "CN=KafkaBroker,O=Pivotal,C=UK" -storepass server-keystore-password > > > -keypass server-key-password > > > > > > keytool -exportcert -file server-cert-file -keystore > server.keystore.jks > > > -alias kafka -storepass server-keystore-password > > > > > > keytool -importcert -file server-cert-file -keystore > > server.truststore.jks > > > -alias kafka -storepass server-truststore-password -noprompt > > > > > > keytool -importcert -file server-cert-file -keystore > > client.truststore.jks > > > -alias kafkaclient -storepass client-truststore-password -noprompt > > > > > > > > > keytool -genkey -alias kafkaclient -keystore client.keystore.jks -dname > > > "CN=KafkaClient,O=Pivotal,C=UK" -storepass client-keystore-password > > > -keypass client-key-password > > > > > > keytool -exportcert -file client-cert-file -keystore > client.keystore.jks > > > -alias kafkaclient -storepass client-keystore-password > > > > > > keytool -importcert -file client-cert-file -keystore > > server.truststore.jks > > > -alias kafkaclient -storepass server-truststore-password -noprompt > > > > > > *Configure broker: Add these lines at the end of your > server.properties* > > > > > > listeners=SSL://:9093 > > > > > > advertised.listeners=SSL://127.0.0.1:9093 > > > > > > ssl.keystore.location=/tmp/acl/server.keystore.jks > > > > > > ssl.keystore.password=server-keystore-password > > > > > > ssl.key.password=server-key-password > > > > > > ssl.truststore.location=/tmp/acl/server.truststore.jks > > > > > > ssl.truststore.password=server-truststore-password > > > > > > security.inter.broker.protocol=SSL > > > > > > security.protocol=SSL > > > > > > ssl.client.auth=required > > > > > > allow.everyone.if.no.acl.found=false > > > > > > authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer > > > > > > super.users=User:CN=KafkaBroker,O=Pivotal,C=UK > > > > > > *Configure producer: producer.properties* > > > > > > security.protocol=SSL > > > > > > ssl.truststore.location=/tmp/acl/client.truststore.jks > > > > > > ssl.truststore.password=client-truststore-password > > > > > > ssl.keystore.location=/tmp/acl/client.keystore.jks > > > > > > ssl.keystore.password=client-keystore-password > > > > > > ssl.key.password=client-key-password > > > > > > > > > *Configure consumer: consumer.properties* > > > > > > security.protocol=SSL > > > > > > ssl.truststore.location=/tmp/acl/client.truststore.jks > > > > > > ssl.truststore.password=client-truststore-password > > > > > > ssl.keystore.location=/tmp/acl/client.keystore.jks > > > > > > ssl.keystore.password=client-keystore-password > > > > > > ssl.key.password=client-key-password > > > > > > group.id=testgroup > > > > > > *Create topic:* > > > > > > bin/kafka-topics.sh --zookeeper localhost --create --topic testtopic > > > --replication-factor 1 --partitions 1 > > > > > > > > > *Configure ACLs:* > > > > > > bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost: > > 2181 > > > --add --allow-principal "User:CN=KafkaClient,O=Pivotal,C=UK" > --producer > > > --topic testtopic > > > > > > bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost: > > 2181 > > > --add --allow-principal "User:CN=KafkaClient,O=Pivotal,C=UK" > --consumer > > > --topic testtopic --group test group > > > > > > > > > *Run console producer and type in some messages:* > > > > > > bin/kafka-console-producer.sh --producer.config > > > /tmp/acl/producer.properties --topic testtopic --broker-list > > > 127.0.0.1:9093 > > > > > > > > > *Run console consumer, you should see messages from above:* > > > > > > > bin/kafka-console-consumer.sh --consumer.config > > > /tmp/acl/consumer.properties --topic testtopic --bootstrap-server > > > 127.0.0.1:9093 --from-beginning > > > > > > > > > > > > On Tue, May 23, 2017 at 12:57 PM, Raghav <raghavas...@gmail.com> > wrote: > > > > > >> Darshan, > > >> > > >> I have not yet successfully gotten the ACLs to work in Kafka. I am > still > > >> looking for help. I will update this email thread if I do find. In > case > > >> you > > >> get it working, please let me know. > > >> > > >> Thanks. > > >> > > >> R > > >> > > >> On Tue, May 23, 2017 at 8:49 AM, Darshan Purandare < > > >> purandare.dars...@gmail.com> wrote: > > >> > > >> > Raghav > > >> > > > >> > I saw few posts of yours around Kafka ACLs and the problems. I have > > seen > > >> > similar issues where Writer has not been able to write to any > topic. I > > >> have > > >> > seen "leader not available" and sometimes "unknown topic or > > partition", > > >> and > > >> > "topic_authorization_failed" error. > > >> > > > >> > Let me know if you find a valid config that works. > > >> > > > >> > Thanks. > > >> > > > >> > > > >> > > > >> > On Tue, May 23, 2017 at 8:44 AM, Raghav <raghavas...@gmail.com> > > wrote: > > >> > > > >> >> Hello Kafka Users > > >> >> > > >> >> I am a new Kafka user and trying to make Kafka SSL work with > > >> Authorization > > >> >> and ACLs. I followed posts from Kafka and Confluent docs exactly to > > the > > >> >> point but my producer cannot write to kafka broker. I get > > >> >> "LEADER_NOT_FOUND" errors. And even Consumer throws the same > errors. > > >> >> > > >> >> Can someone please share their config which worked with ACLs. > > >> >> > > >> >> Here is my config. Please help. > > >> >> > > >> >> server.properties config > > >> >> ------------------------------------------------------------ > > >> >> ------------------------------------------------ > > >> >> broker.id=0 > > >> >> auto.create.topics.enable=true > > >> >> delete.topic.enable=true > > >> >> > > >> >> listeners=PLAINTEXT://kafka1.example.com:9092 > > >> >> <http://kafka-dev1.example.com:9092/>,SSL://kafka1. > example.com:9093 > > >> >> <http://kafka-dev1.example.com:9093/> > > >> >> host.name=kafka1.example.com <http://kafka-dev1.example.com/> > > >> >> > > >> >> > > >> >> > > >> >> ssl.keystore.location=/var/private/kafka1.keystore.jks > > >> >> ssl.keystore.password=12345678 > > >> >> ssl.key.password=12345678 > > >> >> > > >> >> ssl.truststore.location=/var/private/kafka1.truststore.jks > > >> >> ssl.truststore.password=12345678 > > >> >> > > >> >> ssl.client.auth=required > > >> >> ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1 > > >> >> ssl.keystore.type=JKS > > >> >> ssl.truststore.type=JKS > > >> >> > > >> >> authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer > > >> >> ------------------------------------------------------------ > > >> >> ------------------------------------------------ > > >> >> > > >> >> > > >> >> > > >> >> Here is producer Config(producer.properties) > > >> >> ------------------------------------------------------------ > > >> >> ------------------------------------------------ > > >> >> security.protocol=SSL > > >> >> ssl.truststore.location=/var/private/kafka2.truststore.jks > > >> >> ssl.truststore.password=12345678 > > >> >> > > >> >> ssl.keystore.location=/var/private/kafka2.keystore.jks > > >> >> ssl.keystore.password=12345678 > > >> >> ssl.key.password=12345678 > > >> >> > > >> >> ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1 > > >> >> ssl.truststore.type=JKS > > >> >> ssl.keystore.type=JKS > > >> >> > > >> >> ------------------------------------------------------------ > > >> >> ------------------------------------------------ > > >> >> > > >> >> > > >> >> Raqhav > > >> >> > > >> > > > >> > > > >> > > >> > > >> -- > > >> Raghav > > >> > > > > > > > > > > > > -- > > Raghav > > > > > > > > -- > > Raghav > > > > > > -- > > Raghav > > The information contained in this e-mail, and any attachment, is > confidential and is intended solely for the use of the intended recipient. > Access, copying or re-use of the e-mail or any attachment, or any > information contained therein, by any other person is not authorized. If > you are not the intended recipient please return the e-mail to the sender > and delete it from your computer. Although we attempt to sweep e-mail and > attachments for viruses, we do not guarantee that either are virus-free and > accept no liability for any damage sustained as a result of viruses. > > Please refer to http://disclaimer.bnymellon.com/eu.htm for certain > disclosures relating to European legal entities. > -- Raghav
