Re: Infinite loop trying to connect to coordinator

It was on our test environment and nothing was running when the incident occurred. In the server log we have a bunch of [2017-07-11 11:52:15,330] WARN Attempting to send response via channel for which there is no open connection, connection id 0 (kafka.network.Processor) But the time doesn't match

Re: Kafka authorizer ACLs question

Hi SK, Could you please take a look at this document ( https://developer.ibm.com/opentech/2017/05/31/kafka-acls-in-practice/) and confirm you performed the steps in Broker Setup on all brokers? Thanks. --Vahid From: Sruthi Kumar Annamneedu To: users@kafka.apache.org Date: 07/11/2017

Kafka authorizer ACLs question

Hi, I am hoping someone from the community can help me clarify Kafka authorizer feature. *Question:* Do I have to set up any property other than ' authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer' in server.properties file to activate ACLs using Kafka Authorizer? *Background:* We ha

Re: "Failed to update metadata" in integration test for exactly once stream and custom store

Seems Streams cannot connect (or looses connection) to the brokers. Not sure why. You can also have a look here for our own EOS Streams integration test: https://github.com/apache/kafka/blob/trunk/streams/src/test/java/org/apache/kafka/streams/integration/EosIntegrationTest.java -Matthias On 7

Re: Kafka streams: Record linkage / collecting all messages linked to one entity

Hi Wladislaw, Would splitting the one topic into multiple topics be acceptable at all? E.g., you could use the "branch" function in the DSL to split the messages and send to different topics. Then, once you have multiple topics you can do the joins etc. Thoughts? Thanks Eno > On 11 Jul 2017,

"Failed to update metadata" in integration test for exactly once stream and custom store

Hi all, I wrote an integration test to see how Kafka Streams handles exactly once processing with a custom state store, and it's failing with this error: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 6 ms I'm not sure whether this is a bug in my code, a b

Info on ACL - broker, consumer and producer

Hi All I am trying to understand how ACL works between broker, producer and consumer. I am trying to understand internals of ACL to customize it. I browsed through internet and couldn't find much help. Anyone having any documentation or a go to resource would be highly appreciated. Thanks in ad

Consumer group failed with SASL_SSL configuration

Hi All I configured 3 zookeeper and 3 kafka node cluster with SASL_SSL which is working fine. I am able to produce and consume from the command line. But when I try to describe consumer group it fails. Please see below for the command and the error. Any insight is highly appreciated. bin/kaf

How to perform keytool operation using Java code

Hi I followed https://kafka.apache.org/documentation/#security to create keystore and trust store using Java Keytool. Now, I am looking to do the same stuff programmatically using Java. I am struggling to find the right Java classes to perform following operations: 1. How to extract CSR from a ke

Kafka streams: Record linkage / collecting all messages linked to one entity

Hi all. How would one approach the following scenario with Kafka streams? There is one input topic. It has data from different sources in a normalized format. There is a need to join records that come from different sources but are linked to the same entity (record linkage). There is a determini

Re: Infinite loop trying to connect to coordinator

Hi Pierre, Do your brokers remain responsive? In other words, do you see any other symptoms such as decreased write or read throughput which may indicate long GC pauses or possibly heavy load on your zookeeper cluster as evidenced by any SocketTimeoutExceptions on the Kafka and/or Zookeeper sides?

Re: Where to run kafka-consumer-groups.sh from?

Hi, very likely due to timing. What problem is it causing you exactly that you want to work around? These differences shouldn't concern you to much I guess. We use the tool across continents and don't worry about it to much. Offset Commit interval makes everything blury anyways. If you can s

Infinite loop trying to connect to coordinator

Hi, We are using kafka 0.10.2 with 2 brokers and 2 application nodes composed of 6 consumers each (all in one group). And recently we experienced disconnection of both nodes simultaneously and an infinite retry to connect to the coordinator. Currently, just restarting the nodes solve the problem b

Need help regarding Compression

Hi, I need help. We have a Kafka cluster in production with topics compressed with snappy. We want to configure compression.type lz4 on the brokers and restart them (either rolling update, or all down all up if this has benefits). What happens to the existing topics compressed with snappy afte

Re: Consumer vs replication traffic priority

Have you checked the documentation on quotas? https://kafka.apache.org/documentation/#design_quotas This should enable you to do what you're after. __ Sam Pegler PRODUCTION ENGINEER T. +44(0) 07 562 867 486 3-7 Herbal Hill / London / EC1R 5EJ www.infectiousm