Aditya, Yes, you are right. We agreed to think how global config can be done w/o restarting the broker, consumers/producers. I haven't yet updated the KIP accordingly though. In short, the problem is that KafkaConfig is "spread out" over different components, like SocketServer, OffsetManager etc. This makes it hard to dynamically change config, at least if we follow dynamic LogConfig approach. So my opinion and vision is that we need to treat each config block (like quotas in your case) separately. Logically, different cases will be handled differently, some of the configs are not eligible for dynamic change at all.
Thanks, Andrii Biletskyi On Wed, Mar 18, 2015 at 6:54 PM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > Hi Andrii, > > Thanks for the writeup. > IMO, we should be able to support dynamically changing certain configs. > For example: the Quota proposal relies on such a mechanism for changing > quotas on the fly. > > Aditya > ________________________________________ > From: Andrii Biletskyi [andrii.bilets...@stealth.ly] > Sent: Tuesday, March 03, 2015 10:30 AM > To: dev@kafka.apache.org > Subject: Re: [DISCUSS] KIP-5 - Broker Configuration Management > > Hey Jun, > > Initially I was thinking about instead of 3-4 state that global config > changes > take effect only after broker restart. So it's just: > > 3-4. On each broker startup apply global config from ZK > > In other words, the comprehensive workflow is the following: > 1. Issue ChangeGlobalConfigRequest > 2. Controller validates / stores config in ZK > (out of scope of this KIP) 3. Do rolling restart for brokers one by one to > pick up > config changes > > My understanding is that we won't be able to handle config change > dynamically > as we do for Log config. The main reason, from my point of view, is that > broker config operates such settings as num.io.threads updating which would > mean > gracefully restart some of the broker's components (in this case > SocketServer) which is, > in turn, might be very tricky. > > Thanks, > Andrii Biletskyi > > On Tue, Mar 3, 2015 at 7:43 PM, Jun Rao <j...@confluent.io> wrote: > > > It seems the proposed workflow is the following. > > > > 1. Client issues a global config update request to the broker. > > 2. Broker writes the new config to ZK. > > 3. Controller picks up the changes from ZK. > > 4. Controller propagates the config changes to all brokers. > > > > Do we need to add a new request/response to propagate the config changes? > > > > Also, this logic is a bit different from how per topic config changes > > works: each broker reads from ZK directly. It would be good to make them > > consistent. > > > > Thanks, > > > > Jun > > > > > > On Wed, Jan 21, 2015 at 10:34 PM, Joe Stein <joe.st...@stealth.ly> > wrote: > > > > > Created a KIP > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-5+-+Broker+Configuration+Management > > > > > > JIRA https://issues.apache.org/jira/browse/KAFKA-1786 > > > > > > /******************************************* > > > Joe Stein > > > Founder, Principal Consultant > > > Big Data Open Source Security LLC > > > http://www.stealth.ly > > > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > > > ********************************************/ > > > > > >
