I'm running Kafka 1.1.1 and Zookeeper 3.4.6 in a cluster, both guarded by Kerberos. My app stack includes a module containing topic configurations, and my continuous integration build autodeploys changes to topics with kafka-topics.sh and kafka-configs.sh.
When I try to use a non-superuser principal to authenticate in the scripts, the topic metadata is created by kafka-topics.sh in Zookeeper in such a way that Kafka cannot process it to create the actual topics in Kafka brokers - partitions are not created in the broker. Also, running kafka-configs.sh to alter configs of existing topics gets "NoAuth for /configs/<topicname>". When I authenticate with the superuser principal "kafka" then everything works fine. But making the "kafka" superuser credentials available in CI context seems unsecure. Is it possible to use kafka-topics.sh and kafka-configs.sh in a kerberized environment with a non-superuser Kerberos principal and how can this be made to happen? Can you suggest an alternate solution to achieve CI for Kafka topics? Best regards, Kristjan Peil
