for anyone that runs into this. turns out i also had to set:
producer.security.protocol=SASL_PLAINTEXT
producer.sasl.kerberos.service.name=kafka
On Thu, Nov 24, 2016 at 8:54 PM, Koert Kuipers <ko...@tresata.com> wrote:
> i have a secure kafka 0.10.1 cluster using SASL_PLAINTEXT
>
> the kafka servers seem fine, and i can start console-consumer and
> console-producer and i see the message i type in the producer pop up in the
> consumer. no problems so far.
>
> for example to start console-producer:
> $ kinit
> $ export KAFKA_OPTS="-Djava.security.auth.login.config=config/jaas.conf"
> $ bin/kafka-console-producer.sh --producer.config
> config/producer.properties --topic test --broker-list
> SASL_PLAINTEXT://somenode:9092
>
> but i am having no luck whatsoever with kafka-connect. i tried this:
> $ kinit
> $ export KAFKA_OPTS="-Djava.security.auth.login.config=config/jaas.conf"
> $ bin/connect-standalone.sh config/connect-standalone.properties
> config/connect-console-source.properties
>
> my config/connect-console-source.properties is unchanged. my
> config/connect-standalone has:
>
> bootstrap.servers=SASL_PLAINTEXT://somenode:9092
> security.protocol=SASL_PLAINTEXT
> sasl.kerberos.service.name=kafka
> key.converter=org.apache.kafka.connect.json.JsonConverter
> value.converter=org.apache.kafka.connect.json.JsonConverter
> internal.key.converter=org.apache.kafka.connect.json.JsonConverter
> internal.value.converter=org.apache.kafka.connect.json.JsonConverter
> internal.key.converter.schemas.enable=false
> internal.value.converter.schemas.enable=false
> offset.storage.file.filename=/tmp/connect.offsets
> offset.flush.interval.ms=10000
>
> i get these logs in an infinite loop:
> [2016-11-24 20:47:18,528] DEBUG Node -1 disconnected.
> (org.apache.kafka.clients.NetworkClient:463)
> [2016-11-24 20:47:18,528] WARN Bootstrap broker somenode:9092 disconnected
> (org.apache.kafka.clients.NetworkClient:568)
> [2016-11-24 20:47:18,528] DEBUG Give up sending metadata request since no
> node is available (org.apache.kafka.clients.NetworkClient:625)
> [2016-11-24 20:47:18,629] DEBUG Initialize connection to node -1 for
> sending metadata request (org.apache.kafka.clients.NetworkClient:644)
> [2016-11-24 20:47:18,629] DEBUG Initiating connection to node -1 at
> somenode:9092. (org.apache.kafka.clients.NetworkClient:496)
> [2016-11-24 20:47:18,631] DEBUG Created socket with SO_RCVBUF = 32768,
> SO_SNDBUF = 124928, SO_TIMEOUT = 0 to node -1 (org.apache.kafka.common.
> network.Selector:327)
> [2016-11-24 20:47:18,631] DEBUG Completed connection to node -1
> (org.apache.kafka.clients.NetworkClient:476)
> [2016-11-24 20:47:18,730] DEBUG Sending metadata request
> {topics=[connect-test]} to node -1 (org.apache.kafka.clients.
> NetworkClient:640)
> [2016-11-24 20:47:18,730] DEBUG Connection with somenode/192.168.1.54
> disconnected (org.apache.kafka.common.network.Selector:365)
> java.io.EOFException
> at org.apache.kafka.common.network.NetworkReceive.
> readFromReadableChannel(NetworkReceive.java:83)
> at org.apache.kafka.common.network.NetworkReceive.
> readFrom(NetworkReceive.java:71)
> at org.apache.kafka.common.network.KafkaChannel.receive(
> KafkaChannel.java:154)
> at org.apache.kafka.common.network.KafkaChannel.read(
> KafkaChannel.java:135)
> at org.apache.kafka.common.network.Selector.
> pollSelectionKeys(Selector.java:343)
> at org.apache.kafka.common.network.Selector.poll(
> Selector.java:291)
> at org.apache.kafka.clients.NetworkClient.poll(
> NetworkClient.java:260)
> at org.apache.kafka.clients.producer.internals.Sender.run(
> Sender.java:236)
> at org.apache.kafka.clients.producer.internals.Sender.run(
> Sender.java:135)
> at java.lang.Thread.run(Thread.java:745)
>
> i tried different kafka-connect connectors, same result.
>
> any ideas? thanks!
>
