Hello Manikumar,
I appreciate your advice , thank you.
I tried to use SASL_PLAINTEXT with SCRAM enabled hoping that lack of SSL
will help debugging (will switch to SASL_SSL later).
I have 3 brokers running on one box with different ports
listeners = SASL_PLAINTEXT://<MY_IP>:9092
listeners = SASL_PLAINTEXT://<MY_IP>:9093
listeners = SASL_PLAINTEXT://<MY_IP>:9094
0. Changed broker.properties
listeners = SASL_PLAINTEXT://<MY_IP>:9093
sasl.enabled.mechanisms = [SCRAM-SHA-256]
sasl.mechanism.inter.broker.protocol = SCRAM-SHA-256
security.inter.broker.protocol = SASL_PLAINTEXT
1.created admin user for the brokers
bin/kafka-configs.sh --zookeeper localhost:2181 --alter --add-config
'SCRAM-SHA-256=password=admin-secret,SCRAM-SHA-512=password=admin-secret'
--entity-type users --entity-name admin
2.created jaas.conf file in config dir :config/kafka_server_jaas.conf
KafkaServer {
org.apache.kafka.common.security.plain.ScramLoginModule required
username="admin"
password="admin-secret"
user_admin="admin-secret"
user_alice="alice-secret";
};
3. Added export
KAFKA_OPTS="-Djava.security.auth.login.config=config/kafka_server_jaas.conf"
But I can start only one broker, the moment I start second broker I am
getting exceptions like these:
[2017-08-02 04:30:36,733] DEBUG [Replica Manager on Broker 0]: Recording
follower broker 1 log read results:
ArrayBuffer((TNT_GRP_subgroup_getAttributeList_ACK-1,Fetch Data:
[FetchDataInfo(0 [0 : 0],[],false,None)], HW: [0], leaderLogStartOffset:
[0], leaderLogEndOffset: [0], followerLogStartOffset: [0], fetchTimeMs:
[1501648236733], readSize: [1048576], error: [NONE]))
(kafka.server.ReplicaManager)
[2017-08-02 04:30:36,803] DEBUG Accepted connection from /<MY_IP>:58816 on
/<MY_IP>:9093 and assigned it to processor 2, sendBufferSize
[actual|requested]: [102400|102400] recvBufferSize [actual|requested]:
[102400|102400] (kafka.network.Acceptor)
[2017-08-02 04:30:36,803] DEBUG Processor 2 listening to new connection
from /<MY_IP>:58816 (kafka.network.Processor)
[2017-08-02 04:30:36,803] DEBUG Set SASL server state to HANDSHAKE_REQUEST
(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)
[2017-08-02 04:30:36,803] DEBUG Handle Kafka request METADATA
(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)
[2017-08-02 04:30:36,803] DEBUG Set SASL server state to FAILED
(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)
[2017-08-02 04:30:36,803] DEBUG Connection with /<MY_IP> disconnected
(org.apache.kafka.common.network.Selector)
java.io.IOException:
org.apache.kafka.common.errors.IllegalSaslStateException: Unexpected Kafka
request of type METADATA during SASL handshake.
at
org.apache.kafka.common.security.authenticator.SaslServerAuthenticator.authenticate(SaslServerAuthenticator.java:247)
at
org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:76)
at
org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:374)
at org.apache.kafka.common.network.Selector.poll(Selector.java:326)
at kafka.network.Processor.poll(SocketServer.scala:499)
at kafka.network.Processor.run(SocketServer.scala:435)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.kafka.common.errors.IllegalSaslStateException:
Unexpected Kafka request of type METADATA during SASL handshake.
[2017-08-02 04:30:36,905] DEBUG Accepted connection from /<MY_IP>:58823 on
/<MY_IP>:9093 and assigned it to processor 0, sendBufferSize
[actual|requested]: [102400|102400] recvBufferSize [actual|requested]:
[102400|102400] (kafka.network.Acceptor)
[2017-08-02 04:30:36,905] DEBUG Processor 0 listening to new connection
from /<MY_IP>:58823 (kafka.network.Processor)
[2017-08-02 04:30:36,905] DEBUG Set SASL server state to HANDSHAKE_REQUEST
(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)
[2017-08-02 04:30:36,905] DEBUG Handle Kafka request METADATA
(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)
[2017-08-02 04:30:36,905] DEBUG Set SASL server state to FAILED
(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)
[2017-08-02 04:30:36,905] DEBUG Connection with /<MY_IP> disconnected
(org.apache.kafka.common.network.Selector)
java.io.IOException:
org.apache.kafka.common.errors.IllegalSaslStateException: Unexpected Kafka
request of type METADATA during SASL handshake.
Adding separate jaas.conf files for each broker with different users didn't
change anything.
Question - should each broker use separate user for inter broker
communication? Or the reason for exceptions is broker set up on one IP?
Any hints would be highly appreciated.
Thx,
-AL
On Mon, Jul 31, 2017 at 11:08 PM, Manikumar <manikumar.re...@gmail.com>
wrote:
> Server restart is required, only if you are using SASL/PLAIN mechanism.
> Other mechanisms (Kerberos, Scram) restart is not required.
>
> https://issues.apache.org/jira/browse/KAFKA-4292 will help us to write
> custom handlers.
>
> On Tue, Aug 1, 2017 at 4:26 AM, Alexei Levashov <
> alexei.levas...@arrayent.com> wrote:
>
> > Hello,
> >
> > Is there any dynamic approach to add user to the cluster for clients
> > connecting to the running cluster.
> > What I mean by that - can I avoid bouncing a broker if I have to add new
> > user with say SASL authentication?
> > When I add a new entry to kafka_server_jaas.conf it looks like it is
> > required to bounce the broker for changes to take place.
> >
> > Thx,
> > -AL
> >
>
