SASL authentication between Kafka and Zookeeper

[Ivanov, Evgeny]

Hi all,

could you please advise correct configuration settings (jaas and config files) 
to enable SASL authentication between Kafka and Zookeeper ?

Here is the error I get:

[2022-06-06 10:54:30,348] ERROR SASL authentication failed using login context 
'Client'. (org.apache.zookeeper.client.ZooKeeperSaslClient)
javax.security.sasl.SaslException: Error in authenticating with a Zookeeper 
Quorum member: the quorum member's saslToken is null.
                at 
org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslToken(ZooKeeperSaslClient.java:310)
                at 
org.apache.zookeeper.client.ZooKeeperSaslClient.respondToServer(ZooKeeperSaslClient.java:270)
                at 
org.apache.zookeeper.ClientCnxn$SendThread.readResponse(ClientCnxn.java:936)
                at 
org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:98)
                at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
                at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1290)
[2022-06-06 10:54:30,352] DEBUG [ZooKeeperClient Kafka server] Received event: 
WatchedEvent state:AuthFailed type:None path:null 
(kafka.zookeeper.ZooKeeperClient)
[2022-06-06 10:54:30,353] ERROR [ZooKeeperClient Kafka server] Auth failed, 
initialized=true connectionState=AUTH_FAILED (kafka.zookeeper.ZooKeeperClient)
[2022-06-06 10:54:30,365] DEBUG Scheduling task auth-failed with initial delay 
1000 ms and period -1 ms. (kafka.utils.KafkaScheduler)
[2022-06-06 10:54:30,372] INFO EventThread shut down for session: 
0x100ec2b518b0000 (org.apache.zookeeper.ClientCnxn)
[2022-06-06 10:54:30,395] ERROR Fatal error during KafkaServer startup. Prepare 
to shutdown (kafka.server.KafkaServer)
org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode = 
AuthFailed for /consumers
                at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
                at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
                at 
kafka.zookeeper.AsyncResponse.maybeThrow(ZooKeeperClient.scala:566)
                at 
kafka.zk.KafkaZkClient.createRecursive(KafkaZkClient.scala:1729)
                at 
kafka.zk.KafkaZkClient.makeSurePersistentPathExists(KafkaZkClient.scala:1627)
                at 
kafka.zk.KafkaZkClient.$anonfun$createTopLevelPaths$1(KafkaZkClient.scala:1619)
                at 
kafka.zk.KafkaZkClient.$anonfun$createTopLevelPaths$1$adapted(KafkaZkClient.scala:1619)
                at scala.collection.immutable.List.foreach(List.scala:333)
                at 
kafka.zk.KafkaZkClient.createTopLevelPaths(KafkaZkClient.scala:1619)
                at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:492)
                at kafka.server.KafkaServer.startup(KafkaServer.scala:201)
                at kafka.Kafka$.main(Kafka.scala:109)
                at kafka.Kafka.main(Kafka.scala)
[2022-06-06 10:54:30,396] INFO shutting down (kafka.server.KafkaServer)
[2022-06-06 10:54:30,399] INFO [ZooKeeperClient Kafka server] Closing. 
(kafka.zookeeper.ZooKeeperClient)

And here is our configuration:

server.properties

broker.id=432
listeners = PLAINTEXT://smsk02ap432u:9094,SASL_SSL://smsk02ap432u:9095
sasl.enabled.mechanisms=PLAIN
ssl.keystore.location=/app/kafka/certs/server/server.keystore.jks
ssl.keystore.password=Moscow123
authorizer.class.name=kafka.security.authorizer.AclAuthorizer
zookeeper.connect=smsk02ap432u:2181
zookeeper.connection.timeout.ms=18000



kafka_server_jaas.conf

KafkaServer {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="admin-secret"
    user_admin="admin-secret"
    user_alice="alice-secret"
        user_john="John1"
        user_tom="Tom1";
};
Client {
   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="admin"
   password="admin-secret"
   user_admin="admin-secret";
};



zoo.cfg

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/app/zookeeper/data
clientPort=2181
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme=sasl
jaasLoginRenew=3600000



zookeeper_jaas.conf

Server {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="admin-secret"
    user_admin="admin-secret";
};

Best regards,
Evgeny

Business Application Support
VTB Capital
Telephone.: +7 (495) 960 9999 (ext.264423)
Mobile: +7 (916) 091-8939


________________________________

This email message (and any attachments) is confidential and may be privileged 
or otherwise protected from disclosure by applicable law. If you are not the 
intended recipient or have received this in error please notify the system 
manager, postmas...@vtbcapital.ru and remove this message and any attachments 
from your system. Any unauthorized dissemination, copying or other use of this 
message and/or any attachments is strictly prohibited and may constitute a 
breach of civil or criminal law.
JSC VTB Capital may monitor email traffic data and also the content of email.