Re: unable to reconfigure ssl truststore dynamically on broker

2018-08-16 Thread Manikumar
Yes, we have open JIRA for this: https://issues.apache.org/jira/browse/KAFKA-4493 On Wed, Aug 15, 2018 at 10:12 PM John Calcote wrote: > Thanks Manikumar - that's very helpful. I never thought to treat the > AdminClient like the broker or clients and look for a configuration options > set on tha

Re: How to reduce kafka's rebalance time ?

2018-08-16 Thread M. Manna
You have been recommended to upgrade to a newer version of Kafka, or tune timeout params. Adhering to a older version is more of the users’ decision. Perhaps, we should simply put older versions as “End of Life”. As part of open source initiative, you are always welcome to debug and demonstrate ho

Re: How to reduce kafka's rebalance time ?

2018-08-16 Thread Shantanu Deshmukh
Hi Manna, I meant no offense. Simply meant to say that haven't found solution to my problem from here. Apologies, if my sentence was off the line. On Thu, Aug 16, 2018 at 4:05 PM M. Manna wrote: > You have been recommended to upgrade to a newer version of Kafka, or tune > timeout params. Adheri

Re: Very long consumer rebalances

2018-08-16 Thread Shantanu Deshmukh
I saw a few topics with segment.ms and retention.ms property set. Can that be causing any issue? I remember that this is the only change I carried out to the cluster in last couple of months after which the problem started. On Fri, Aug 10, 2018 at 2:55 PM M. Manna wrote: > if you can upgrade, I

Interest in automatic topology/rack awareness?

2018-08-16 Thread Michael Gasch
Hi, Today, the configuration of rack awareness for brokers (broker.rack=my-rack-id) is either manual or via a script during deployment of a broker. Other systems, like Hadoop or Kubernetes, have support for auto-detection of topologies, often called well-known labels like racks, zones or region

Re: How to reduce kafka's rebalance time ?

2018-08-16 Thread Steve Tian
My $0.02: 1. Read the documentation. 2. Help people to understand your problem better: Try to describe your problem in a gist and see if you can provide some details like: which version of Kafka you're using on client/server side, your consumer/producer/server code/configuration that can reproduce

Please help: Zookeeper not coming up after power down

2018-08-16 Thread Raghav
Hi Our 3 node Zookeeper ensemble got powered down, and upon powering up the zookeeper could get quorum and kept throwing these errors. As a result our Kafka cluster was unusable. What is the best way to revive ZK cluster in such situations ? Please suggest. 2018-08-17_00:59:18.87009 2018-08-17 0

Re: Please help: Zookeeper not coming up after power down

2018-08-16 Thread Dan Simoes
Ensure ids/nodes are correct in zoo.cfg and zookeeper is running on each. Also any changes to ports being open? If it’s aws, beck security groups. Node 1 cannot talk to the other two nodes. > On Aug 16, 2018, at 6:02 PM, Raghav wrote: > > Hi > > Our 3 node Zookeeper ensemble got powered d

?????? How to reduce kafka's rebalance time ?

2018-08-16 Thread ????de????
I modified some consumer configuration items, mainly about the parameters of fetch frequency, heartbeat, and session timeout. The problem of long time rebalance has not been found in the test environment for a long time. The relevant configuration is as follows:fetch-max-wait: 1s heartbeat-inter

Re: How to reduce kafka's rebalance time ?

2018-08-16 Thread Steve Tian
Interesting. Thanks for letting us know. On Fri, 17 Aug 2018 at 11:22 堅強de泡沫 wrote: > I modified some consumer configuration items, mainly about the parameters > of fetch frequency, heartbeat, and session timeout. The problem of long > time rebalance has not been found in the test environment f