In SSL cert, there is a field which has a CN (Common Name). So when ACLs are set, they are set for that CN. This is how the ACLs are configured and matched against. I am still pretty new to Kafka in general, but this is how I think it works. I can copy my config if you want.
On Thu, May 25, 2017 at 12:51 PM, Mike Marzo <precisionarchery...@gmail.com> wrote: > Stupid question.... > If u don't specify a jaas file how does the consumer and producer specify > the Id that acl's are configured against.... boy I am getting more and > more perplexed by this... > > mike marzo > 908 209-4484 <(908)%20209-4484> > > On May 24, 2017 9:29 PM, "Raghav" <raghavas...@gmail.com> wrote: > >> Mike >> >> I am not using jaas file. I literally took the config Rajini gave in the >> previous email and it worked for me. I am using ssl Kafka with ACLs. I am >> not suing kerberos. >> >> Thanks. >> >> On Wed, May 24, 2017 at 11:29 AM, Mike Marzo < >> precisionarchery...@gmail.com> wrote: >> >>> I'm also having issues getting acls to work. Out of intereat, are you >>> starting ur brokers with a jaas file, if so do u mind sharing the client >>> and server side jaas entries so I can validate what I'm doing. >>> >>> mike marzo >>> 908 209-4484 >>> >>> On May 24, 2017 10:54 AM, "Raghav" <raghavas...@gmail.com> wrote: >>> >>> > Hi Rajini >>> > >>> > Thank you very much. It perfectly works. >>> > >>> > I think in my setup I was trying to use a CA (certificate authority) to >>> > sign the certificates from client and server, and then adding it to >>> trust >>> > store and keystore. I think in that process, I may have messed >>> something. I >>> > will try above config with a CA to sign certificates. Hopefully that >>> would >>> > work too. >>> > >>> > Thanks a lot again. >>> > >>> > Raghav >>> > >>> > >>> > >>> > >>> > On Wed, May 24, 2017 at 7:04 AM, Rajini Sivaram < >>> rajinisiva...@gmail.com> >>> > wrote: >>> > >>> > > Raghav/Darshan, >>> > > >>> > > Can you try these steps on a clean installation of Kafka? It works >>> for >>> > me, >>> > > so hopefully it will work for you. And then you can adapt to your >>> > scenario. >>> > > >>> > > *Create keystores and truststores:* >>> > > >>> > > keytool -genkey -alias kafka -keystore server.keystore.jks -dname >>> > > "CN=KafkaBroker,O=Pivotal,C=UK" -storepass server-keystore-password >>> > > -keypass server-key-password >>> > > >>> > > keytool -exportcert -file server-cert-file -keystore >>> server.keystore.jks >>> > > -alias kafka -storepass server-keystore-password >>> > > >>> > > keytool -importcert -file server-cert-file -keystore >>> > server.truststore.jks >>> > > -alias kafka -storepass server-truststore-password -noprompt >>> > > >>> > > keytool -importcert -file server-cert-file -keystore >>> > client.truststore.jks >>> > > -alias kafkaclient -storepass client-truststore-password -noprompt >>> > > >>> > > >>> > > keytool -genkey -alias kafkaclient -keystore client.keystore.jks >>> -dname >>> > > "CN=KafkaClient,O=Pivotal,C=UK" -storepass client-keystore-password >>> > > -keypass client-key-password >>> > > >>> > > keytool -exportcert -file client-cert-file -keystore >>> client.keystore.jks >>> > > -alias kafkaclient -storepass client-keystore-password >>> > > >>> > > keytool -importcert -file client-cert-file -keystore >>> > server.truststore.jks >>> > > -alias kafkaclient -storepass server-truststore-password -noprompt >>> > > >>> > > *Configure broker: Add these lines at the end of your >>> server.properties* >>> > > >>> > > listeners=SSL://:9093 >>> > > >>> > > advertised.listeners=SSL://127.0.0.1:9093 >>> > > >>> > > ssl.keystore.location=/tmp/acl/server.keystore.jks >>> > > >>> > > ssl.keystore.password=server-keystore-password >>> > > >>> > > ssl.key.password=server-key-password >>> > > >>> > > ssl.truststore.location=/tmp/acl/server.truststore.jks >>> > > >>> > > ssl.truststore.password=server-truststore-password >>> > > >>> > > security.inter.broker.protocol=SSL >>> > > >>> > > security.protocol=SSL >>> > > >>> > > ssl.client.auth=required >>> > > >>> > > allow.everyone.if.no.acl.found=false >>> > > >>> > > authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer >>> > > >>> > > super.users=User:CN=KafkaBroker,O=Pivotal,C=UK >>> > > >>> > > *Configure producer: producer.properties* >>> > > >>> > > security.protocol=SSL >>> > > >>> > > ssl.truststore.location=/tmp/acl/client.truststore.jks >>> > > >>> > > ssl.truststore.password=client-truststore-password >>> > > >>> > > ssl.keystore.location=/tmp/acl/client.keystore.jks >>> > > >>> > > ssl.keystore.password=client-keystore-password >>> > > >>> > > ssl.key.password=client-key-password >>> > > >>> > > >>> > > *Configure consumer: consumer.properties* >>> > > >>> > > security.protocol=SSL >>> > > >>> > > ssl.truststore.location=/tmp/acl/client.truststore.jks >>> > > >>> > > ssl.truststore.password=client-truststore-password >>> > > >>> > > ssl.keystore.location=/tmp/acl/client.keystore.jks >>> > > >>> > > ssl.keystore.password=client-keystore-password >>> > > >>> > > ssl.key.password=client-key-password >>> > > >>> > > group.id=testgroup >>> > > >>> > > *Create topic:* >>> > > >>> > > bin/kafka-topics.sh --zookeeper localhost --create --topic testtopic >>> > > --replication-factor 1 --partitions 1 >>> > > >>> > > >>> > > *Configure ACLs:* >>> > > >>> > > bin/kafka-acls.sh --authorizer-properties >>> zookeeper.connect=localhost: >>> > 2181 >>> > > --add --allow-principal "User:CN=KafkaClient,O=Pivotal,C=UK" >>> --producer >>> > > --topic testtopic >>> > > >>> > > bin/kafka-acls.sh --authorizer-properties >>> zookeeper.connect=localhost: >>> > 2181 >>> > > --add --allow-principal "User:CN=KafkaClient,O=Pivotal,C=UK" >>> --consumer >>> > > --topic testtopic --group test group >>> > > >>> > > >>> > > *Run console producer and type in some messages:* >>> > > >>> > > bin/kafka-console-producer.sh --producer.config >>> > > /tmp/acl/producer.properties --topic testtopic --broker-list >>> > > 127.0.0.1:9093 >>> > > >>> > > >>> > > *Run console consumer, you should see messages from above:* >>> > > >>> > > bin/kafka-console-consumer.sh --consumer.config >>> > > /tmp/acl/consumer.properties --topic testtopic --bootstrap-server >>> > > 127.0.0.1:9093 --from-beginning >>> > > >>> > > >>> > > >>> > > On Tue, May 23, 2017 at 12:57 PM, Raghav <raghavas...@gmail.com> >>> wrote: >>> > > >>> > >> Darshan, >>> > >> >>> > >> I have not yet successfully gotten the ACLs to work in Kafka. I am >>> still >>> > >> looking for help. I will update this email thread if I do find. In >>> case >>> > >> you >>> > >> get it working, please let me know. >>> > >> >>> > >> Thanks. >>> > >> >>> > >> R >>> > >> >>> > >> On Tue, May 23, 2017 at 8:49 AM, Darshan Purandare < >>> > >> purandare.dars...@gmail.com> wrote: >>> > >> >>> > >> > Raghav >>> > >> > >>> > >> > I saw few posts of yours around Kafka ACLs and the problems. I >>> have >>> > seen >>> > >> > similar issues where Writer has not been able to write to any >>> topic. I >>> > >> have >>> > >> > seen "leader not available" and sometimes "unknown topic or >>> > partition", >>> > >> and >>> > >> > "topic_authorization_failed" error. >>> > >> > >>> > >> > Let me know if you find a valid config that works. >>> > >> > >>> > >> > Thanks. >>> > >> > >>> > >> > >>> > >> > >>> > >> > On Tue, May 23, 2017 at 8:44 AM, Raghav <raghavas...@gmail.com> >>> > wrote: >>> > >> > >>> > >> >> Hello Kafka Users >>> > >> >> >>> > >> >> I am a new Kafka user and trying to make Kafka SSL work with >>> > >> Authorization >>> > >> >> and ACLs. I followed posts from Kafka and Confluent docs exactly >>> to >>> > the >>> > >> >> point but my producer cannot write to kafka broker. I get >>> > >> >> "LEADER_NOT_FOUND" errors. And even Consumer throws the same >>> errors. >>> > >> >> >>> > >> >> Can someone please share their config which worked with ACLs. >>> > >> >> >>> > >> >> Here is my config. Please help. >>> > >> >> >>> > >> >> server.properties config >>> > >> >> ------------------------------------------------------------ >>> > >> >> ------------------------------------------------ >>> > >> >> broker.id=0 >>> > >> >> auto.create.topics.enable=true >>> > >> >> delete.topic.enable=true >>> > >> >> >>> > >> >> listeners=PLAINTEXT://kafka1.example.com:9092 >>> > >> >> <http://kafka-dev1.example.com:9092/>,SSL://kafka1.example.c >>> om:9093 >>> > >> >> <http://kafka-dev1.example.com:9093/> >>> > >> >> host.name=kafka1.example.com <http://kafka-dev1.example.com/> >>> > >> >> >>> > >> >> >>> > >> >> >>> > >> >> ssl.keystore.location=/var/private/kafka1.keystore.jks >>> > >> >> ssl.keystore.password=12345678 >>> > >> >> ssl.key.password=12345678 >>> > >> >> >>> > >> >> ssl.truststore.location=/var/private/kafka1.truststore.jks >>> > >> >> ssl.truststore.password=12345678 >>> > >> >> >>> > >> >> ssl.client.auth=required >>> > >> >> ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1 >>> > >> >> ssl.keystore.type=JKS >>> > >> >> ssl.truststore.type=JKS >>> > >> >> >>> > >> >> authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer >>> > >> >> ------------------------------------------------------------ >>> > >> >> ------------------------------------------------ >>> > >> >> >>> > >> >> >>> > >> >> >>> > >> >> Here is producer Config(producer.properties) >>> > >> >> ------------------------------------------------------------ >>> > >> >> ------------------------------------------------ >>> > >> >> security.protocol=SSL >>> > >> >> ssl.truststore.location=/var/private/kafka2.truststore.jks >>> > >> >> ssl.truststore.password=12345678 >>> > >> >> >>> > >> >> ssl.keystore.location=/var/private/kafka2.keystore.jks >>> > >> >> ssl.keystore.password=12345678 >>> > >> >> ssl.key.password=12345678 >>> > >> >> >>> > >> >> ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1 >>> > >> >> ssl.truststore.type=JKS >>> > >> >> ssl.keystore.type=JKS >>> > >> >> >>> > >> >> ------------------------------------------------------------ >>> > >> >> ------------------------------------------------ >>> > >> >> >>> > >> >> >>> > >> >> Raqhav >>> > >> >> >>> > >> > >>> > >> > >>> > >> >>> > >> >>> > >> -- >>> > >> Raghav >>> > >> >>> > > >>> > > >>> > >>> > >>> > -- >>> > Raghav >>> > >>> >> >> >> >> -- >> Raghav >> > -- Raghav
