My bad it should be under Client section Client { com.sun.security.auth.module.Krb5LoginModule required debug=true useKeyTab=true storeKey=true serviceName=zookeeper keyTab="/home/oleg/kafka_2.10-0.9.0.1/config/security/kafka.keytab" principal="kafka/ubuntu.oleg....@oleg.com"; };
-Harsha On Tue, Feb 23, 2016, at 11:37 AM, Harsha wrote: > can you try adding "serviceName=zookeeper" to KafkaServer section like > KafkaServer { > com.sun.security.auth.module.Krb5LoginModule required > debug=true > useKeyTab=true > storeKey=true > serviceName=zookeeper > keyTab="/home/oleg/kafka_2.10-0.9.0.1/config/security/kafka.keytab" > principal="kafka/ubuntu.oleg....@oleg.com"; > }; > > On Tue, Feb 23, 2016, at 11:24 AM, Oleg Zhurakousky wrote: > > More info > > > > I am starting both services as myself ‘oleg’. Validated that both key tab > > files are readable. o I am assuming Zookeeper is started as ‘zookeeper’ > > and Kafka as ‘kafka’ > > > > Oleg > > > > > On Feb 23, 2016, at 2:22 PM, Oleg Zhurakousky > > > <ozhurakou...@hortonworks.com> wrote: > > > > > > Harsha > > > > > > Thanks for following up. Here is is: > > > oleg@ubuntu:~/kafka_2.10-0.9.0.1/config$ cat kafka_server_jaas.conf > > > KafkaServer { > > > com.sun.security.auth.module.Krb5LoginModule required > > > debug=true > > > useKeyTab=true > > > storeKey=true > > > keyTab="/home/oleg/kafka_2.10-0.9.0.1/config/security/kafka.keytab" > > > principal="kafka/ubuntu.oleg....@oleg.com"; > > > }; > > > Client { > > > com.sun.security.auth.module.Krb5LoginModule required > > > debug=true > > > useKeyTab=true > > > storeKey=true > > > keyTab="/home/oleg/kafka_2.10-0.9.0.1/config/security/kafka.keytab" > > > principal="kafka/ubuntu.oleg....@oleg.com"; > > > }; > > > > > > oleg@ubuntu:~/kafka_2.10-0.9.0.1/config$ cat zookeeper_jaas.conf > > > Server { > > > com.sun.security.auth.module.Krb5LoginModule required > > > debug=true > > > useKeyTab=true > > > keyTab="/home/oleg/kafka_2.10-0.9.0.1/config/security/zookeeper.keytab" > > > storeKey=true > > > useTicketCache=false > > > principal="zookeeper/ubuntu.oleg....@oleg.com"; > > > }; > > > > > > Cheers > > > Oleg > > > > > >> On Feb 23, 2016, at 2:17 PM, Harsha <ka...@harsha.io> wrote: > > >> > > >> Oleg, > > >> Can you post your jaas configs. Its important that serviceName > > >> must match the principal name with which zookeeper is running. > > >> Whats the principal name zookeeper service is running with. > > >> -Harsha > > >> > > >> On Tue, Feb 23, 2016, at 11:01 AM, Oleg Zhurakousky wrote: > > >>> Hey guys, first post here so bare with me > > >>> > > >>> Trying to setup Kerberized Kafka 0.9.0.. Followed the instructions here > > >>> http://kafka.apache.org/documentation.html#security_sasl and i seem to > > >>> be > > >>> very close, but not quite there yet. > > >>> > > >>> ZOOKEEPER > > >>> Starting Zookeeper seems to be OK (below is the relevant part of the > > >>> log) > > >>> . . . > > >>> [2016-02-23 13:22:40,336] INFO maxSessionTimeout set to -1 > > >>> (org.apache.zookeeper.server.ZooKeeperServer) > > >>> Debug is true storeKey true useTicketCache false useKeyTab true > > >>> doNotPrompt false ticketCache is null isInitiator true KeyTab is > > >>> /home/oleg/kafka_2.10-0.9.0.1/config/security/zookeeper.keytab > > >>> refreshKrb5Config is false principal is > > >>> zookeeper/ubuntu.oleg....@oleg.com<mailto:zookeeper/ubuntu.oleg....@oleg.com> > > >>> tryFirstPass is false useFirstPass is false storePass is false clearPass > > >>> is false > > >>> principal is > > >>> zookeeper/ubuntu.oleg....@oleg.com<mailto:zookeeper/ubuntu.oleg....@oleg.com> > > >>> Will use keytab > > >>> Commit Succeeded > > >>> > > >>> [2016-02-23 13:22:40,541] INFO successfully logged in. > > >>> (org.apache.zookeeper.Login) > > >>> [2016-02-23 13:22:40,544] INFO binding to port 0.0.0.0/0.0.0.0:2181 > > >>> (org.apache.zookeeper.server.NIOServerCnxnFactory) > > >>> [2016-02-23 13:22:40,544] INFO TGT refresh thread started. > > >>> (org.apache.zookeeper.Login) > > >>> [2016-02-23 13:22:40,554] INFO TGT valid starting at: Tue Feb 23 > > >>> 13:22:40 EST 2016 (org.apache.zookeeper.Login) > > >>> [2016-02-23 13:22:40,554] INFO TGT expires: Tue Feb 23 > > >>> 23:22:40 EST 2016 (org.apache.zookeeper.Login) > > >>> [2016-02-23 13:22:40,554] INFO TGT refresh sleeping until: Tue Feb 23 > > >>> 21:47:35 EST 2016 (org.apache.zookeeper.Login) > > >>> [2016-02-23 13:23:09,012] INFO Accepted socket connection from > > >>> /127.0.0.1:51876 (org.apache.zookeeper.server.NIOServerCnxnFactory) > > >>> [2016-02-23 13:23:09,025] INFO Client attempting to establish new > > >>> session > > >>> at /127.0.0.1:51876 (org.apache.zookeeper.server.ZooKeeperServer) > > >>> [2016-02-23 13:23:09,026] INFO Creating new log file: log.57 > > >>> (org.apache.zookeeper.server.persistence.FileTxnLog) > > >>> . . . > > >>> > > >>> > > >>> KAFKA > > >>> Starting Kafka server is not going well yet although I see that > > >>> interaction with Kerberos is successful (see relevant log below. the > > >>> error is at the bottom) > > >>> . . . > > >>> [2016-02-23 13:26:11,508] INFO starting (kafka.server.KafkaServer) > > >>> [2016-02-23 13:26:11,511] INFO Connecting to zookeeper on localhost:2181 > > >>> (kafka.server.KafkaServer) > > >>> [2016-02-23 13:26:11,519] INFO JAAS File name: > > >>> /home/oleg/kafka_2.10-0.9.0.1/config/kafka_server_jaas.conf > > >>> (org.I0Itec.zkclient.ZkClient) > > >>> [2016-02-23 13:26:11,520] INFO Starting ZkClient event thread. > > >>> (org.I0Itec.zkclient.ZkEventThread) > > >>> [2016-02-23 13:26:11,527] INFO Client > > >>> environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 > > >>> GMT (org.apache.zookeeper.ZooKeeper) > > >>> [2016-02-23 13:26:11,527] INFO Client > > >>> environment:host.name=172.16.137.20 > > >>> (org.apache.zookeeper.ZooKeeper) > > >>> [2016-02-23 13:26:11,527] INFO Client environment:java.version=1.8.0_72 > > >>> (org.apache.zookeeper.ZooKeeper) > > >>> [2016-02-23 13:26:11,527] INFO Client environment:java.vendor=Oracle > > >>> Corporation (org.apache.zookeeper.ZooKeeper) > > >>> [2016-02-23 13:26:11,527] INFO Client > > >>> environment:java.home=/usr/lib/jvm/java-8-oracle/jre > > >>> (org.apache.zookeeper.ZooKeeper) > > >>> [2016-02-23 13:26:11,527] INFO Client > > >>> environment:java.class.path=:/home/oleg/kafka_2.10-0.9.0.1/bin/../libs/jetty-http-9.2.12.v20150709.jar:/home/oleg/ka. > > >>> . . . . . > > >>> [2016-02-23 13:26:11,531] INFO Client > > >>> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib > > >>> (org.apache.zookeeper.ZooKeeper) > > >>> [2016-02-23 13:26:11,531] INFO Client environment:java.io.tmpdir=/tmp > > >>> (org.apache.zookeeper.ZooKeeper) > > >>> [2016-02-23 13:26:11,531] INFO Client environment:java.compiler=<NA> > > >>> (org.apache.zookeeper.ZooKeeper) > > >>> [2016-02-23 13:26:11,531] INFO Client environment:os.name=Linux > > >>> (org.apache.zookeeper.ZooKeeper) > > >>> [2016-02-23 13:26:11,531] INFO Client environment:os.arch=amd64 > > >>> (org.apache.zookeeper.ZooKeeper) > > >>> [2016-02-23 13:26:11,531] INFO Client > > >>> environment:os.version=4.2.0-27-generic (org.apache.zookeeper.ZooKeeper) > > >>> [2016-02-23 13:26:11,531] INFO Client environment:user.name=oleg > > >>> (org.apache.zookeeper.ZooKeeper) > > >>> [2016-02-23 13:26:11,531] INFO Client environment:user.home=/home/oleg > > >>> (org.apache.zookeeper.ZooKeeper) > > >>> [2016-02-23 13:26:11,531] INFO Client > > >>> environment:user.dir=/home/oleg/kafka_2.10-0.9.0.1 > > >>> (org.apache.zookeeper.ZooKeeper) > > >>> [2016-02-23 13:26:11,532] INFO Initiating client connection, > > >>> connectString=localhost:2181 sessionTimeout=6000 > > >>> watcher=org.I0Itec.zkclient.ZkClient@647fd8ce > > >>> (org.apache.zookeeper.ZooKeeper) > > >>> [2016-02-23 13:26:11,541] INFO Waiting for keeper state > > >>> SaslAuthenticated > > >>> (org.I0Itec.zkclient.ZkClient) > > >>> Debug is true storeKey true useTicketCache false useKeyTab true > > >>> doNotPrompt false ticketCache is null isInitiator true KeyTab is > > >>> /home/oleg/kafka_2.10-0.9.0.1/config/security/kafka.keytab > > >>> refreshKrb5Config is false principal is > > >>> kafka/ubuntu.oleg....@oleg.com<mailto:kafka/ubuntu.oleg....@oleg.com> > > >>> tryFirstPass is false useFirstPass is false storePass is false clearPass > > >>> is false > > >>> principal is > > >>> kafka/ubuntu.oleg....@oleg.com<mailto:kafka/ubuntu.oleg....@oleg.com> > > >>> Will use keytab > > >>> Commit Succeeded > > >>> > > >>> [2016-02-23 13:26:11,734] INFO successfully logged in. > > >>> (org.apache.zookeeper.Login) > > >>> [2016-02-23 13:26:11,735] INFO TGT refresh thread started. > > >>> (org.apache.zookeeper.Login) > > >>> [2016-02-23 13:26:11,738] INFO Client will use GSSAPI as SASL mechanism. > > >>> (org.apache.zookeeper.client.ZooKeeperSaslClient) > > >>> [2016-02-23 13:26:11,743] INFO Opening socket connection to server > > >>> localhost/127.0.0.1:2181. Will attempt to SASL-authenticate using Login > > >>> Context section 'Client' (org.apache.zookeeper.ClientCnxn) > > >>> [2016-02-23 13:26:11,748] INFO Socket connection established to > > >>> localhost/127.0.0.1:2181, initiating session > > >>> (org.apache.zookeeper.ClientCnxn) > > >>> [2016-02-23 13:26:11,752] INFO TGT valid starting at: Tue Feb 23 > > >>> 13:26:11 EST 2016 (org.apache.zookeeper.Login) > > >>> [2016-02-23 13:26:11,752] INFO TGT expires: Tue Feb 23 > > >>> 23:26:11 EST 2016 (org.apache.zookeeper.Login) > > >>> [2016-02-23 13:26:11,752] INFO TGT refresh sleeping until: Tue Feb 23 > > >>> 21:40:22 EST 2016 (org.apache.zookeeper.Login) > > >>> [2016-02-23 13:26:11,761] INFO Session establishment complete on server > > >>> localhost/127.0.0.1:2181, sessionid = 0x1530f5e6fcb0001, negotiated > > >>> timeout = 6000 (org.apache.zookeeper.ClientCnxn) > > >>> [2016-02-23 13:26:11,762] INFO zookeeper state changed (SyncConnected) > > >>> (org.I0Itec.zkclient.ZkClient) > > >>> [2016-02-23 13:26:11,773] ERROR An error: > > >>> (java.security.PrivilegedActionException: > > >>> javax.security.sasl.SaslException: GSS initiate failed [Caused by > > >>> GSSException: No valid credentials provided (Mechanism level: Server not > > >>> found in Kerberos database (7) - LOOKING_UP_SERVER)]) occurred when > > >>> evaluating Zookeeper Quorum Member's received SASL token. Zookeeper > > >>> Client will go to AUTH_FAILED state. > > >>> (org.apache.zookeeper.client.ZooKeeperSaslClient) > > >>> [2016-02-23 13:26:11,773] ERROR SASL authentication with Zookeeper > > >>> Quorum > > >>> member failed: javax.security.sasl.SaslException: An error: > > >>> (java.security.PrivilegedActionException: > > >>> javax.security.sasl.SaslException: GSS initiate failed [Caused by > > >>> GSSException: No valid credentials provided (Mechanism level: Server not > > >>> found in Kerberos database (7) - LOOKING_UP_SERVER)]) occurred when > > >>> evaluating Zookeeper Quorum Member's received SASL token. Zookeeper > > >>> Client will go to AUTH_FAILED state. (org.apache.zookeeper.ClientCnxn) > > >>> [2016-02-23 13:26:11,774] INFO zookeeper state changed (AuthFailed) > > >>> (org.I0Itec.zkclient.ZkClient) > > >>> [2016-02-23 13:26:17,542] INFO Terminate ZkClient event thread. > > >>> (org.I0Itec.zkclient.ZkEventThread) > > >>> . . . > > >>> > > >>> Any pointers? > > >>> > > >>> Cheers > > >>> Oleg > > >>> > > >> > > > > > > > >