I see a couple of things that look wrong. First, remove this line from your
ZK config:
zookeeper.authProvider.1=org.apache.zookeeper.server.auth.DigestAuthenticationProvider.
And replace it with this:
authProvider.sasl=org.apache.zookeeper.server.auth.SASLAuthenticationProvider.
Additionally, I think you need to add these lines to the ZK config if you
want ZK to ZK authentication:
quorum.auth.enableSasl=true
quorum.auth.learnerRequireSasl=true
quorum.auth.serverRequireSasl=true
The rest looks OK.
On Sat, Nov 11, 2023 at 9:12 PM arjun s v <arjun.cs...@gmail.com> wrote:
> A small correction, I'm not trying to enable mTLS, just a simple
> authentication(Digest or Plain) is enough,
> Sharing the jaas files and config files,
> kafka_server_jaas.conf
>
> > admin.KafkaServer{
> >
> > org.apache.kafka.common.security.plain.PlainLoginModule required
> >
> > username="USERNAME"
> >
> > password="PASSWORD";
> >
> > };
> >
> > KafkaServer{
> >
> > org.apache.kafka.common.security.plain.PlainLoginModule required
> >
> > username="USERNAME"
> >
> > password="PASSWORD";
> >
> > };
> >
> > Client{
> >
> > org.apache.zookeeper.server.auth.DigestLoginModule required
> >
> > username="super"
> >
> > password="adminsecret";
> >
> > };
> >
> server properties
>
> > zookeeper.sasl.client=false
>
> java.security.auth.login.config=$base_dir/../config/kafka_server_jaas.conf
>
> zookeeper.connection.timeout.ms=6000
>
> zookeeper.sync.time.ms=2000
>
> zookeeper.set.acl=true
>
>
> zk_server_jaas.conf
>
> > QuorumServer{
> > org.apache.zookeeper.server.auth.DigestLoginModule required
> > user_test="test";
> > };
> > QuorumLearner{
> > org.apache.zookeeper.server.auth.DigestLoginModule required
> > username="test"
> > password="test";
> > };
> > Server{
> > org.apache.zookeeper.server.auth.DigestLoginModule required
> > username="super"
> > password="adminsecret";
>
> };
>
>
> zoo.cfg
>
> >
> >
> zookeeper.authProvider.1=org.apache.zookeeper.server.auth.DigestAuthenticationProvider
>
>
> env
>
> > SERVER_JVMFLAGS="$SERVER_JVMFLAGS
> > -Djava.security.auth.login.config=$ZOOCFGDIR/zk_server_jaas.conf"
> >
> On Fri, Nov 10, 2023 at 7:03 PM Alex Brekken <brek...@gmail.com> wrote:
>
> > Ok, so you're trying to enable both SASL authentication (digest) and TLS,
> > using mTLS for Zookeeper? I'm just trying to understand the bigger
> > picture. The error you're getting regarding the Sasl token sounds like
> > either the jaas config on the Kafka broker side is wrong/missing, or the
> > jaas config on the ZK side is wrong/missing. (you need both - in this
> case
> > the broker is the "client" and ZK is the "server"). Are you able to share
> > the jaas config you're using for both Kafka and ZK? Without seeing that
> > it's tough to know. Also, to make troubleshooting easier you might want
> to
> > leave TLS out of it for now and get SASL working first. (or vice-versa)
> >
> >
> >
> > On Thu, Nov 9, 2023 at 11:26 PM arjun s v <arjun.cs...@gmail.com> wrote:
> >
> > > "Digest-MD5 is SASL authentication, so not sure what you mean here."
> > > If I set zookeeper.sasl.client=true, zookeeper expects a "saslToken"
> and
> > > throws the following error,
> > >
> > > "SASL authentication failed using login context 'Client' with
> exception:
> > > {}" "javax.security.sasl.SaslException: Error in authenticating with a
> > > Zookeeper Quorum member: the quorum member's saslToken is null.
> > >
> > > at
> > >
> > >
> >
> org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslToken(ZooKeeperSaslClient.java:312)
> > >
> > > at
> > >
> > >
> >
> org.apache.zookeeper.client.ZooKeeperSaslClient.respondToServer(ZooKeeperSaslClient.java:275)
> > >
> > > at
> > >
> > >
> >
> org.apache.zookeeper.ClientCnxn$SendThread.readResponse(ClientCnxn.java:882)
> > >
> > > at
> > >
> >
> org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:103)
> > >
> > > at
> > >
> > >
> >
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:365)
> > >
> > > at
> > > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223)
> > >
> > >
> > > "Hmm, that config shouldn't have anything to do with TLS. You can set
> > ACL's
> > >
> > > with or without TLS encryption. Were you getting an error?"
> > >
> > >
> > > "Fatal error during KafkaServer startup. Prepare to shutdown"
> > > "java.lang.SecurityException: zookeeper.set.acl is true, but ZooKeeper
> > > client TLS configuration identifying at least
> > > kafka.server.KafkaConfig$@7b22ec89.ZkSslClientEnableProp,
> > > kafka.server.KafkaConfig$@7b22ec89.ZkClientCnxnSocketProp, and
> > > kafka.server.KafkaConfig$@7b22ec89.ZkSslKeyStoreLocationProp was not
> > > present and the verification of the JAAS login file failed
> > > [java.security.auth.login.config=./../config/kafka_server_jaas.conf,
> > > zookeeper.sasl.client=false,
> zookeeper.sasl.clientconfig=default:Client]
> > >
> > > at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:445)
> > >
> > > at kafka.server.KafkaServer.startup(KafkaServer.scala:191)
> > >
> > > at kafka.Kafka$.main(Kafka.scala:109)
> > >
> > > at kafka.Kafka.main(Kafka.scala)
> > >
> > >
> > > "This was the 2nd result in a google search:
> > > https://docs.confluent.io/platform/current/security/zk-security.html";
> > >
> > > FYKI, I've googled, asked chat gpt, surfed over many zookeeper and
> kafka
> > > docs and blog,
> > > I remember trying the doc you suggested here about 10 days back in the
> > > initial days of this task!
> > > About the doc you suggested,
> > > I cannot configure SSL as I already mentioned, If I skip ssl config
> part
> > > from your suggested doc and tried Digest-MD5, I come up "saslToken
> > missing"
> > > exception which I mentioned above!
> > > I don't really understand what saslToken is and how to make it get
> > > generated for Digest auth!
> > > Please assist!
> > >
> > > On Thu, Nov 9, 2023 at 7:15 PM Alex Craig <alexcrai...@gmail.com>
> wrote:
> > >
> > > > " I couldn't find any doc by kafka to enable Digest-MD5
> > authentication."
> > > > This was the 2nd result in a google search:
> > > > https://docs.confluent.io/platform/current/security/zk-security.html
> > > >
> > > > " I don't want to enable SASL."
> > > > Digest-MD5 is SASL authentication, so not sure what you mean here.
> > > >
> > > > " If I set zookeeper.set.acl=true, I'm forced to configure TLS."
> > > > Hmm, that config shouldn't have anything to do with TLS. You can set
> > > ACL's
> > > > with or without TLS encryption. Were you getting an error?
> > > >
> > > > On Wed, Nov 8, 2023 at 11:35 PM arjun s v <arjun.cs...@gmail.com>
> > wrote:
> > > >
> > > > > Team,
> > > > >
> > > > > Please consider this as high priority, we need to enable
> > authentication
> > > > > ASAP. Please assist.
> > > > > On Tue, Nov 7, 2023 at 4:38 PM arjun s v <arjun.cs...@gmail.com>
> > > wrote:
> > > > >
> > > > > > Hi team,
> > > > > >
> > > > > > I'm trying to configure *Digest-MD5* authentication between kafka
> > and
> > > > > > zookeeper.
> > > > > > Also I need to set ACL with digest scheme and credentials.
> > > > > > I don't want to enable SASL.
> > > > > > I tried to follow this
> > > > > > <
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Client-Server+mutual+authentication
> > > > >
> > > > > doc
> > > > > > from zookeeper,
> > > > > >
> > > > > > - If I configured a jaas file, I have to set
> > > > > zookeeper.sasl.client=true(if
> > > > > > not kafka throws error from JaasUtils) which enables sasl
> > > > > authentication.
> > > > > > - If I set zookeeper.set.acl=true, I'm forced to configure
> TLS.
> > > > > >
> > > > > > I couldn't find any doc by kafka to enable Digest-MD5
> > authentication.
> > > > > > I cannot configure kerberos or TLS, just a Digest-MD5 is
> sufficient
> > > for
> > > > > my
> > > > > > usecase.
> > > > > > Please let me know if there are any docs to enable Digest-MD5
> auth
> > > > > between
> > > > > > kafka and zookeeper.
> > > > > >
> > > > > > Regards,
> > > > > > Arjun S V
> > > > > >
> > > > >
> > > >
> > >
> >
>
