Stupid question.... If u don't specify a jaas file how does the consumer and producer specify the Id that acl's are configured against.... boy I am getting more and more perplexed by this...
mike marzo 908 209-4484 On May 24, 2017 9:29 PM, "Raghav" <raghavas...@gmail.com> wrote: > Mike > > I am not using jaas file. I literally took the config Rajini gave in the > previous email and it worked for me. I am using ssl Kafka with ACLs. I am > not suing kerberos. > > Thanks. > > On Wed, May 24, 2017 at 11:29 AM, Mike Marzo < > precisionarchery...@gmail.com> wrote: > >> I'm also having issues getting acls to work. Out of intereat, are you >> starting ur brokers with a jaas file, if so do u mind sharing the client >> and server side jaas entries so I can validate what I'm doing. >> >> mike marzo >> 908 209-4484 >> >> On May 24, 2017 10:54 AM, "Raghav" <raghavas...@gmail.com> wrote: >> >> > Hi Rajini >> > >> > Thank you very much. It perfectly works. >> > >> > I think in my setup I was trying to use a CA (certificate authority) to >> > sign the certificates from client and server, and then adding it to >> trust >> > store and keystore. I think in that process, I may have messed >> something. I >> > will try above config with a CA to sign certificates. Hopefully that >> would >> > work too. >> > >> > Thanks a lot again. >> > >> > Raghav >> > >> > >> > >> > >> > On Wed, May 24, 2017 at 7:04 AM, Rajini Sivaram < >> rajinisiva...@gmail.com> >> > wrote: >> > >> > > Raghav/Darshan, >> > > >> > > Can you try these steps on a clean installation of Kafka? It works for >> > me, >> > > so hopefully it will work for you. And then you can adapt to your >> > scenario. >> > > >> > > *Create keystores and truststores:* >> > > >> > > keytool -genkey -alias kafka -keystore server.keystore.jks -dname >> > > "CN=KafkaBroker,O=Pivotal,C=UK" -storepass server-keystore-password >> > > -keypass server-key-password >> > > >> > > keytool -exportcert -file server-cert-file -keystore >> server.keystore.jks >> > > -alias kafka -storepass server-keystore-password >> > > >> > > keytool -importcert -file server-cert-file -keystore >> > server.truststore.jks >> > > -alias kafka -storepass server-truststore-password -noprompt >> > > >> > > keytool -importcert -file server-cert-file -keystore >> > client.truststore.jks >> > > -alias kafkaclient -storepass client-truststore-password -noprompt >> > > >> > > >> > > keytool -genkey -alias kafkaclient -keystore client.keystore.jks >> -dname >> > > "CN=KafkaClient,O=Pivotal,C=UK" -storepass client-keystore-password >> > > -keypass client-key-password >> > > >> > > keytool -exportcert -file client-cert-file -keystore >> client.keystore.jks >> > > -alias kafkaclient -storepass client-keystore-password >> > > >> > > keytool -importcert -file client-cert-file -keystore >> > server.truststore.jks >> > > -alias kafkaclient -storepass server-truststore-password -noprompt >> > > >> > > *Configure broker: Add these lines at the end of your >> server.properties* >> > > >> > > listeners=SSL://:9093 >> > > >> > > advertised.listeners=SSL://127.0.0.1:9093 >> > > >> > > ssl.keystore.location=/tmp/acl/server.keystore.jks >> > > >> > > ssl.keystore.password=server-keystore-password >> > > >> > > ssl.key.password=server-key-password >> > > >> > > ssl.truststore.location=/tmp/acl/server.truststore.jks >> > > >> > > ssl.truststore.password=server-truststore-password >> > > >> > > security.inter.broker.protocol=SSL >> > > >> > > security.protocol=SSL >> > > >> > > ssl.client.auth=required >> > > >> > > allow.everyone.if.no.acl.found=false >> > > >> > > authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer >> > > >> > > super.users=User:CN=KafkaBroker,O=Pivotal,C=UK >> > > >> > > *Configure producer: producer.properties* >> > > >> > > security.protocol=SSL >> > > >> > > ssl.truststore.location=/tmp/acl/client.truststore.jks >> > > >> > > ssl.truststore.password=client-truststore-password >> > > >> > > ssl.keystore.location=/tmp/acl/client.keystore.jks >> > > >> > > ssl.keystore.password=client-keystore-password >> > > >> > > ssl.key.password=client-key-password >> > > >> > > >> > > *Configure consumer: consumer.properties* >> > > >> > > security.protocol=SSL >> > > >> > > ssl.truststore.location=/tmp/acl/client.truststore.jks >> > > >> > > ssl.truststore.password=client-truststore-password >> > > >> > > ssl.keystore.location=/tmp/acl/client.keystore.jks >> > > >> > > ssl.keystore.password=client-keystore-password >> > > >> > > ssl.key.password=client-key-password >> > > >> > > group.id=testgroup >> > > >> > > *Create topic:* >> > > >> > > bin/kafka-topics.sh --zookeeper localhost --create --topic testtopic >> > > --replication-factor 1 --partitions 1 >> > > >> > > >> > > *Configure ACLs:* >> > > >> > > bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost: >> > 2181 >> > > --add --allow-principal "User:CN=KafkaClient,O=Pivotal,C=UK" >> --producer >> > > --topic testtopic >> > > >> > > bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost: >> > 2181 >> > > --add --allow-principal "User:CN=KafkaClient,O=Pivotal,C=UK" >> --consumer >> > > --topic testtopic --group test group >> > > >> > > >> > > *Run console producer and type in some messages:* >> > > >> > > bin/kafka-console-producer.sh --producer.config >> > > /tmp/acl/producer.properties --topic testtopic --broker-list >> > > 127.0.0.1:9093 >> > > >> > > >> > > *Run console consumer, you should see messages from above:* >> > > >> > > bin/kafka-console-consumer.sh --consumer.config >> > > /tmp/acl/consumer.properties --topic testtopic --bootstrap-server >> > > 127.0.0.1:9093 --from-beginning >> > > >> > > >> > > >> > > On Tue, May 23, 2017 at 12:57 PM, Raghav <raghavas...@gmail.com> >> wrote: >> > > >> > >> Darshan, >> > >> >> > >> I have not yet successfully gotten the ACLs to work in Kafka. I am >> still >> > >> looking for help. I will update this email thread if I do find. In >> case >> > >> you >> > >> get it working, please let me know. >> > >> >> > >> Thanks. >> > >> >> > >> R >> > >> >> > >> On Tue, May 23, 2017 at 8:49 AM, Darshan Purandare < >> > >> purandare.dars...@gmail.com> wrote: >> > >> >> > >> > Raghav >> > >> > >> > >> > I saw few posts of yours around Kafka ACLs and the problems. I have >> > seen >> > >> > similar issues where Writer has not been able to write to any >> topic. I >> > >> have >> > >> > seen "leader not available" and sometimes "unknown topic or >> > partition", >> > >> and >> > >> > "topic_authorization_failed" error. >> > >> > >> > >> > Let me know if you find a valid config that works. >> > >> > >> > >> > Thanks. >> > >> > >> > >> > >> > >> > >> > >> > On Tue, May 23, 2017 at 8:44 AM, Raghav <raghavas...@gmail.com> >> > wrote: >> > >> > >> > >> >> Hello Kafka Users >> > >> >> >> > >> >> I am a new Kafka user and trying to make Kafka SSL work with >> > >> Authorization >> > >> >> and ACLs. I followed posts from Kafka and Confluent docs exactly >> to >> > the >> > >> >> point but my producer cannot write to kafka broker. I get >> > >> >> "LEADER_NOT_FOUND" errors. And even Consumer throws the same >> errors. >> > >> >> >> > >> >> Can someone please share their config which worked with ACLs. >> > >> >> >> > >> >> Here is my config. Please help. >> > >> >> >> > >> >> server.properties config >> > >> >> ------------------------------------------------------------ >> > >> >> ------------------------------------------------ >> > >> >> broker.id=0 >> > >> >> auto.create.topics.enable=true >> > >> >> delete.topic.enable=true >> > >> >> >> > >> >> listeners=PLAINTEXT://kafka1.example.com:9092 >> > >> >> <http://kafka-dev1.example.com:9092/>,SSL://kafka1.example. >> com:9093 >> > >> >> <http://kafka-dev1.example.com:9093/> >> > >> >> host.name=kafka1.example.com <http://kafka-dev1.example.com/> >> > >> >> >> > >> >> >> > >> >> >> > >> >> ssl.keystore.location=/var/private/kafka1.keystore.jks >> > >> >> ssl.keystore.password=12345678 >> > >> >> ssl.key.password=12345678 >> > >> >> >> > >> >> ssl.truststore.location=/var/private/kafka1.truststore.jks >> > >> >> ssl.truststore.password=12345678 >> > >> >> >> > >> >> ssl.client.auth=required >> > >> >> ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1 >> > >> >> ssl.keystore.type=JKS >> > >> >> ssl.truststore.type=JKS >> > >> >> >> > >> >> authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer >> > >> >> ------------------------------------------------------------ >> > >> >> ------------------------------------------------ >> > >> >> >> > >> >> >> > >> >> >> > >> >> Here is producer Config(producer.properties) >> > >> >> ------------------------------------------------------------ >> > >> >> ------------------------------------------------ >> > >> >> security.protocol=SSL >> > >> >> ssl.truststore.location=/var/private/kafka2.truststore.jks >> > >> >> ssl.truststore.password=12345678 >> > >> >> >> > >> >> ssl.keystore.location=/var/private/kafka2.keystore.jks >> > >> >> ssl.keystore.password=12345678 >> > >> >> ssl.key.password=12345678 >> > >> >> >> > >> >> ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1 >> > >> >> ssl.truststore.type=JKS >> > >> >> ssl.keystore.type=JKS >> > >> >> >> > >> >> ------------------------------------------------------------ >> > >> >> ------------------------------------------------ >> > >> >> >> > >> >> >> > >> >> Raqhav >> > >> >> >> > >> > >> > >> > >> > >> >> > >> >> > >> -- >> > >> Raghav >> > >> >> > > >> > > >> > >> > >> > -- >> > Raghav >> > >> > > > > -- > Raghav >
