Yes we did. I just overlooked that line.. cleaning it up now. Aditya
________________________________________ From: Gwen Shapira [gshap...@cloudera.com] Sent: Friday, May 15, 2015 12:55 PM To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-21 Configuration Management The wiki says: "There will be 3 paths within config /config/clients/<client_id> /config/topics/<topic_name> /config/brokers/<broker_id> Didn't we decide that brokers will not be configured dynamically, rather we will keep the config in the file? On Fri, May 15, 2015 at 10:46 PM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > Updated the wiki to capture our recent discussions. Please read. > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration > > Thanks, > Aditya > > ________________________________________ > From: Joel Koshy [jjkosh...@gmail.com] > Sent: Tuesday, May 12, 2015 1:09 PM > To: dev@kafka.apache.org > Subject: Re: [DISCUSS] KIP-21 Configuration Management > > The lack of audit could be addressed to some degree by an internal > __config_changes topic which can have very long retention. Also, per > the hangout summary that Gwen sent out it appears that we decided > against supporting SIGHUP/dynamic configs for the broker. > > Thanks, > > Joel > > On Tue, May 12, 2015 at 11:05:06AM -0700, Neha Narkhede wrote: > > Thanks for chiming in, Todd! > > > > Agree that the lack of audit and rollback is a major downside of moving > all > > configs to ZooKeeper. Being able to configure dynamically created > entities > > in Kafka is required though. So I think what Todd suggested is a good > > solution to managing all configs - catching SIGHUP for broker configs and > > storing dynamic configs in ZK like we do today. > > > > On Tue, May 12, 2015 at 10:30 AM, Jay Kreps <jay.kr...@gmail.com> wrote: > > > > > Hmm, here is how I think we can change the split brain proposal to > make it > > > a bit better: > > > 1. Get rid of broker overrides, this is just done in the config file. > This > > > makes the precedence chain a lot clearer (e.g. zk always overrides > file on > > > a per-entity basis). > > > 2. Get rid of the notion of dynamic configs in ConfigDef and in the > broker. > > > All overrides are dynamic and all server configs are static. > > > 3. Create an equivalent of LogConfig for ClientConfig and any future > config > > > type we make. > > > 4. Generalize the TopicConfigManager to handle multiple types of > overrides. > > > > > > What we haven't done is try to think through how the pure zk approach > would > > > work. > > > > > > -Jay > > > > > > > > > > > > On Mon, May 11, 2015 at 10:53 PM, Ashish Singh <asi...@cloudera.com> > > > wrote: > > > > > > > I agree with the Joel's suggestion on keeping broker's configs in > > > > config file and clients/topics config in ZK. Few other projects, > Apache > > > > Solr for one, also does something similar for its configurations. > > > > > > > > On Monday, May 11, 2015, Gwen Shapira <gshap...@cloudera.com> wrote: > > > > > > > > > I like this approach (obviously). > > > > > I am also OK with supporting broker re-read of config file based > on ZK > > > > > watch instead of SIGHUP, if we see this as more consistent with the > > > rest > > > > of > > > > > our code base. > > > > > > > > > > Either is fine by me as long as brokers keep the file and just do > > > refresh > > > > > :) > > > > > > > > > > On Tue, May 12, 2015 at 2:54 AM, Joel Koshy <jjkosh...@gmail.com > > > > > <javascript:;>> wrote: > > > > > > > > > > > So the general concern here is the dichotomy of configs (which we > > > > > > already have - i.e., in the form of broker config files vs topic > > > > > > configs in zookeeper). We (at LinkedIn) had some discussions on > this > > > > > > last week and had this very question for the operations team > whose > > > > > > opinion is I think to a large degree a touchstone for this > decision: > > > > > > "Has the operations team at LinkedIn experienced any pain so far > with > > > > > > managing topic configs in ZooKeeper (while broker configs are > > > > > > file-based)?" It turns out that ops overwhelmingly favors the > current > > > > > > approach. i.e., service configs as file-based configs and > > > client/topic > > > > > > configs in ZooKeeper is intuitive and works great. This may be > > > > > > somewhat counter-intuitive to devs, but this is one of those > > > decisions > > > > > > for which ops input is very critical - because for all practical > > > > > > purposes, they are the users in this discussion. > > > > > > > > > > > > If we continue with this dichotomy and need to support dynamic > config > > > > > > for client/topic configs as well as select service configs then > there > > > > > > will need to be dichotomy in the config change mechanism as well. > > > > > > i.e., client/topic configs will change via (say) a ZooKeeper > watch > > > and > > > > > > the service config will change via a config file re-read (on > SIGHUP) > > > > > > after config changes have been pushed out to local files. Is > this a > > > > > > bad thing? Personally, I don't think it is - i.e. I'm in favor of > > > this > > > > > > approach. What do others think? > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Joel > > > > > > > > > > > > On Mon, May 11, 2015 at 11:08:44PM +0300, Gwen Shapira wrote: > > > > > > > What Todd said :) > > > > > > > > > > > > > > (I think my ops background is showing...) > > > > > > > > > > > > > > On Mon, May 11, 2015 at 10:17 PM, Todd Palino < > tpal...@gmail.com > > > > > <javascript:;>> wrote: > > > > > > > > > > > > > > > I understand your point here, Jay, but I disagree that we > can't > > > > have > > > > > > two > > > > > > > > configuration systems. We have two different types of > > > configuration > > > > > > > > information. We have configuration that relates to the > service > > > > itself > > > > > > (the > > > > > > > > Kafka broker), and we have configuration that relates to the > > > > content > > > > > > within > > > > > > > > the service (topics). I would put the client configuration > > > (quotas) > > > > > in > > > > > > the > > > > > > > > with the second part, as it is dynamic information. I just > don't > > > > see > > > > > a > > > > > > good > > > > > > > > argument for effectively degrading the configuration for the > > > > service > > > > > > > > because of trying to keep it paired with the configuration of > > > > dynamic > > > > > > > > resources. > > > > > > > > > > > > > > > > -Todd > > > > > > > > > > > > > > > > On Mon, May 11, 2015 at 11:33 AM, Jay Kreps < > jay.kr...@gmail.com > > > > > <javascript:;>> > > > > > > wrote: > > > > > > > > > > > > > > > > > I totally agree that ZK is not in-and-of-itself a > configuration > > > > > > > > management > > > > > > > > > solution and it would be better if we could just keep all > our > > > > > config > > > > > > in > > > > > > > > > files. Anyone who has followed the various config > discussions > > > > over > > > > > > the > > > > > > > > past > > > > > > > > > few years of discussion knows I'm the biggest proponent of > > > > > immutable > > > > > > > > > file-driven config. > > > > > > > > > > > > > > > > > > The analogy to "normal unix services" isn't actually quite > > > right > > > > > > though. > > > > > > > > > The problem Kafka has is that a number of the configurable > > > > entities > > > > > > it > > > > > > > > > manages are added dynamically--topics, clients, consumer > > > groups, > > > > > etc. > > > > > > > > What > > > > > > > > > this actually resembles is not a unix services like HTTPD > but a > > > > > > database, > > > > > > > > > and databases typically do manage config dynamically for > > > exactly > > > > > the > > > > > > same > > > > > > > > > reason. > > > > > > > > > > > > > > > > > > The last few emails are arguing that files > ZK as a config > > > > > > solution. I > > > > > > > > > agree with this, but that isn't really the question, > right?The > > > > > > reality is > > > > > > > > > that we need to be able to configure dynamically created > > > entities > > > > > > and we > > > > > > > > > won't get a satisfactory solution to that using files (e.g. > > > rsync > > > > > is > > > > > > not > > > > > > > > an > > > > > > > > > acceptable topic creation mechanism). What we are > discussing is > > > > > > having a > > > > > > > > > single config mechanism or multiple. If we have multiple > you > > > need > > > > > to > > > > > > > > solve > > > > > > > > > the whole config lifecycle problem for both--management, > audit, > > > > > > rollback, > > > > > > > > > etc. > > > > > > > > > > > > > > > > > > Gwen, you were saying we couldn't get rid of the > configuration > > > > > file, > > > > > > not > > > > > > > > > sure if I understand. Is that because we need to give the > URL > > > for > > > > > ZK? > > > > > > > > > Wouldn't the same argument work to say that we can't use > > > > > > configuration > > > > > > > > > files because we have to specify the file path? I think we > can > > > > just > > > > > > give > > > > > > > > > the server the same --zookeeper argument we use everywhere > > > else, > > > > > > right? > > > > > > > > > > > > > > > > > > -Jay > > > > > > > > > > > > > > > > > > On Sun, May 10, 2015 at 11:28 AM, Todd Palino < > > > tpal...@gmail.com > > > > > <javascript:;>> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > I've been watching this discussion for a while, and I > have to > > > > > jump > > > > > > in > > > > > > > > and > > > > > > > > > > side with Gwen here. I see no benefit to putting the > configs > > > > into > > > > > > > > > Zookeeper > > > > > > > > > > entirely, and a lot of downside. The two biggest > problems I > > > > have > > > > > > with > > > > > > > > > this > > > > > > > > > > are: > > > > > > > > > > > > > > > > > > > > 1) Configuration management. OK, so you can write glue > for > > > Chef > > > > > to > > > > > > put > > > > > > > > > > configs into Zookeeper. You also need to write glue for > > > Puppet. > > > > > And > > > > > > > > > > Cfengine. And everything else out there. Files are an > > > industry > > > > > > standard > > > > > > > > > > practice, they're how just about everyone handles it, and > > > > there's > > > > > > > > reasons > > > > > > > > > > for that, not just "it's the way it's always been done". > > > > > > > > > > > > > > > > > > > > 2) Auditing. Configuration files can easily be managed > in a > > > > > source > > > > > > > > > > repository system which tracks what changes were made > and who > > > > > made > > > > > > > > them. > > > > > > > > > It > > > > > > > > > > also easily allows for rolling back to a previous > version. > > > > > > Zookeeper > > > > > > > > does > > > > > > > > > > not. > > > > > > > > > > > > > > > > > > > > I see absolutely nothing wrong with putting the quota > > > (client) > > > > > > configs > > > > > > > > > and > > > > > > > > > > the topic config overrides in Zookeeper, and keeping > > > everything > > > > > > else > > > > > > > > > > exactly where it is, in the configuration file. To handle > > > > > > > > configurations > > > > > > > > > > for the broker that can be changed at runtime without a > > > > restart, > > > > > > you > > > > > > > > can > > > > > > > > > > use the industry standard practice of catching SIGHUP and > > > > > > rereading the > > > > > > > > > > configuration file at that point. > > > > > > > > > > > > > > > > > > > > -Todd > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, May 10, 2015 at 4:00 AM, Gwen Shapira < > > > > > > gshap...@cloudera.com <javascript:;>> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > I am still not clear about the benefits of managing > > > > > > configuration in > > > > > > > > > > > ZooKeeper vs. keeping the local file and adding a > "refresh" > > > > > > mechanism > > > > > > > > > > > (signal, protocol, zookeeper, or other). > > > > > > > > > > > > > > > > > > > > > > Benefits of staying with configuration file: > > > > > > > > > > > 1. In line with pretty much any Linux service that > exists, > > > so > > > > > > admins > > > > > > > > > > have a > > > > > > > > > > > lot of related experience. > > > > > > > > > > > 2. Much smaller change to our code-base, so easier to > > > patch, > > > > > > review > > > > > > > > and > > > > > > > > > > > test. Lower risk overall. > > > > > > > > > > > > > > > > > > > > > > Can you walk me over the benefits of using Zookeeper? > > > > > Especially > > > > > > > > since > > > > > > > > > it > > > > > > > > > > > looks like we can't get rid of the file entirely? > > > > > > > > > > > > > > > > > > > > > > Gwen > > > > > > > > > > > > > > > > > > > > > > On Thu, May 7, 2015 at 3:33 AM, Jun Rao < > j...@confluent.io > > > > > <javascript:;>> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > One of the Chef users confirmed that Chef integration > > > could > > > > > > still > > > > > > > > > work > > > > > > > > > > if > > > > > > > > > > > > all configs are moved to ZK. My rough understanding > of > > > how > > > > > Chef > > > > > > > > works > > > > > > > > > > is > > > > > > > > > > > > that a user first registers a service host with a > Chef > > > > > server. > > > > > > > > After > > > > > > > > > > > that, > > > > > > > > > > > > a Chef client will be run on the service host. The > user > > > can > > > > > > then > > > > > > > > push > > > > > > > > > > > > config changes intended for a service/host to the > Chef > > > > > server. > > > > > > The > > > > > > > > > > server > > > > > > > > > > > > is then responsible for pushing the changes to Chef > > > > clients. > > > > > > Chef > > > > > > > > > > clients > > > > > > > > > > > > support pluggable logic. For example, it can > generate a > > > > > config > > > > > > file > > > > > > > > > > that > > > > > > > > > > > > Kafka broker will take. If we move all configs to > ZK, we > > > > can > > > > > > > > > customize > > > > > > > > > > > the > > > > > > > > > > > > Chef client to use our config CLI to make the config > > > > changes > > > > > in > > > > > > > > > Kafka. > > > > > > > > > > In > > > > > > > > > > > > this model, one probably doesn't need to register > every > > > > > broker > > > > > > in > > > > > > > > > Chef > > > > > > > > > > > for > > > > > > > > > > > > the config push. Not sure if Puppet works in a > similar > > > way. > > > > > > > > > > > > > > > > > > > > > > > > Also for storing the configs, we probably can't > store the > > > > > > > > > broker/global > > > > > > > > > > > > level configs in Kafka itself (e.g. in a special > topic). > > > > The > > > > > > reason > > > > > > > > > is > > > > > > > > > > > that > > > > > > > > > > > > in order to start a broker, we likely need to make > some > > > > > broker > > > > > > > > level > > > > > > > > > > > config > > > > > > > > > > > > changes (e.g., the default log.dir may not be > present, > > > the > > > > > > default > > > > > > > > > port > > > > > > > > > > > may > > > > > > > > > > > > not be available, etc). If we need a broker to be up > to > > > > make > > > > > > those > > > > > > > > > > > changes, > > > > > > > > > > > > we get into this chicken and egg problem. > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > > > > On Tue, May 5, 2015 at 4:14 PM, Gwen Shapira < > > > > > > > > gshap...@cloudera.com <javascript:;>> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Sorry I missed the call today :) > > > > > > > > > > > > > > > > > > > > > > > > > > I think an additional requirement would be: > > > > > > > > > > > > > Make sure that traditional deployment tools > (Puppet, > > > > Chef, > > > > > > etc) > > > > > > > > are > > > > > > > > > > > still > > > > > > > > > > > > > capable of managing Kafka configuration. > > > > > > > > > > > > > > > > > > > > > > > > > > For this reason, I'd like the configuration > refresh to > > > be > > > > > > pretty > > > > > > > > > > close > > > > > > > > > > > to > > > > > > > > > > > > > what most Linux services are doing to force a > reload of > > > > > > > > > > configuration. > > > > > > > > > > > > > AFAIK, this involves handling HUP signal in the > main > > > > thread > > > > > > to > > > > > > > > > reload > > > > > > > > > > > > > configuration. Then packaging scripts can add > something > > > > > nice > > > > > > like > > > > > > > > > > > > "service > > > > > > > > > > > > > kafka reload". > > > > > > > > > > > > > > > > > > > > > > > > > > (See Apache web server: > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/httpd/blob/trunk/build/rpm/httpd.init#L101 > > > > > > > > > > ) > > > > > > > > > > > > > > > > > > > > > > > > > > Gwen > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, May 5, 2015 at 8:54 AM, Joel Koshy < > > > > > > jjkosh...@gmail.com <javascript:;>> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Good discussion. Since we will be talking about > this > > > at > > > > > > 11am, I > > > > > > > > > > > wanted > > > > > > > > > > > > > > to organize these comments into requirements to > see > > > if > > > > we > > > > > > are > > > > > > > > all > > > > > > > > > > on > > > > > > > > > > > > > > the same page. > > > > > > > > > > > > > > > > > > > > > > > > > > > > REQUIREMENT 1: Needs to accept dynamic config > > > changes. > > > > > This > > > > > > > > needs > > > > > > > > > > to > > > > > > > > > > > > > > be general enough to work for all configs that we > > > > > envision > > > > > > may > > > > > > > > > need > > > > > > > > > > > to > > > > > > > > > > > > > > accept changes at runtime. e.g., log (topic), > broker, > > > > > > client > > > > > > > > > > > (quotas), > > > > > > > > > > > > > > etc.. possible options include: > > > > > > > > > > > > > > - ZooKeeper watcher > > > > > > > > > > > > > > - Kafka topic > > > > > > > > > > > > > > - Direct RPC to controller (or config > coordinator) > > > > > > > > > > > > > > > > > > > > > > > > > > > > The current KIP is really focused on REQUIREMENT > 1 > > > and > > > > I > > > > > > think > > > > > > > > > that > > > > > > > > > > > is > > > > > > > > > > > > > > reasonable as long as we don't come up with > something > > > > > that > > > > > > > > > requires > > > > > > > > > > > > > > significant re-engineering to support the other > > > > > > requirements. > > > > > > > > > > > > > > > > > > > > > > > > > > > > REQUIREMENT 2: Provide consistency of configs > across > > > > > > brokers > > > > > > > > > > (modulo > > > > > > > > > > > > > > per-broker overrides) or at least be able to > verify > > > > > > > > consistency. > > > > > > > > > > > What > > > > > > > > > > > > > > this effectively means is that config changes > must be > > > > > seen > > > > > > by > > > > > > > > all > > > > > > > > > > > > > > brokers eventually and we should be able to > easily > > > > > compare > > > > > > the > > > > > > > > > full > > > > > > > > > > > > > > config of each broker. > > > > > > > > > > > > > > > > > > > > > > > > > > > > REQUIREMENT 3: Central config store. Needs to > work > > > with > > > > > > plain > > > > > > > > > > > > > > file-based configs and other systems (e.g., > puppet). > > > > > > Ideally, > > > > > > > > > > should > > > > > > > > > > > > > > not bring in other dependencies (e.g., a DB). > > > Possible > > > > > > options: > > > > > > > > > > > > > > - ZooKeeper > > > > > > > > > > > > > > - Kafka topic > > > > > > > > > > > > > > - other? E.g. making it pluggable? > > > > > > > > > > > > > > > > > > > > > > > > > > > > Any other requirements? > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > Joel > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, May 05, 2015 at 01:38:09AM +0000, Aditya > > > > Auradkar > > > > > > > > wrote: > > > > > > > > > > > > > > > Hey Neha, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the feedback. > > > > > > > > > > > > > > > 1. In my earlier exchange with Jay, I > mentioned the > > > > > > broker > > > > > > > > > > writing > > > > > > > > > > > > all > > > > > > > > > > > > > > it's configs to ZK (while respecting the > overrides). > > > > Then > > > > > > ZK > > > > > > > > can > > > > > > > > > be > > > > > > > > > > > > used > > > > > > > > > > > > > to > > > > > > > > > > > > > > view all configs. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2. Need to think about this a bit more. > Perhaps we > > > > can > > > > > > > > discuss > > > > > > > > > > this > > > > > > > > > > > > > > during the hangout tomorrow? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 3 & 4) I viewed these config changes as mainly > > > > > > administrative > > > > > > > > > > > > > > operations. In the case, it may be reasonable to > > > assume > > > > > > that > > > > > > > > the > > > > > > > > > ZK > > > > > > > > > > > > port > > > > > > > > > > > > > is > > > > > > > > > > > > > > available for communication from the machine > these > > > > > > commands are > > > > > > > > > > run. > > > > > > > > > > > > > Having > > > > > > > > > > > > > > a ConfigChangeRequest (or similar) is nice to > have > > > but > > > > > > having a > > > > > > > > > new > > > > > > > > > > > API > > > > > > > > > > > > > and > > > > > > > > > > > > > > sending requests to controller also change how > we do > > > > > topic > > > > > > > > based > > > > > > > > > > > > > > configuration currently. I was hoping to keep > this > > > KIP > > > > as > > > > > > > > minimal > > > > > > > > > > as > > > > > > > > > > > > > > possible and provide a means to represent and > modify > > > > > > client and > > > > > > > > > > > broker > > > > > > > > > > > > > > based configs in a central place. Are there any > > > > concerns > > > > > > if we > > > > > > > > > > tackle > > > > > > > > > > > > > these > > > > > > > > > > > > > > things in a later KIP? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > Aditya > > > > > > > > > > > > > > > ________________________________________ > > > > > > > > > > > > > > > From: Neha Narkhede [n...@confluent.io > > > > <javascript:;>] > > > > > > > > > > > > > > > Sent: Sunday, May 03, 2015 9:48 AM > > > > > > > > > > > > > > > To: dev@kafka.apache.org <javascript:;> > > > > > > > > > > > > > > > Subject: Re: [DISCUSS] KIP-21 Configuration > > > > Management > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for starting this discussion, Aditya. > Few > > > > > > > > > > questions/comments > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 1. If you change the default values like it's > > > > mentioned > > > > > > in > > > > > > > > the > > > > > > > > > > KIP, > > > > > > > > > > > > do > > > > > > > > > > > > > > you > > > > > > > > > > > > > > > also overwrite the local config file as part of > > > > > updating > > > > > > the > > > > > > > > > > > default > > > > > > > > > > > > > > value? > > > > > > > > > > > > > > > If not, where does the admin look to find the > > > default > > > > > > values, > > > > > > > > > ZK > > > > > > > > > > or > > > > > > > > > > > > > local > > > > > > > > > > > > > > > Kafka config file? What if a config value is > > > > different > > > > > in > > > > > > > > both > > > > > > > > > > > > places? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2. I share Gwen's concern around making sure > that > > > > > popular > > > > > > > > > config > > > > > > > > > > > > > > management > > > > > > > > > > > > > > > tools continue to work with this change. Would > love > > > > to > > > > > > see > > > > > > > > how > > > > > > > > > > each > > > > > > > > > > > > of > > > > > > > > > > > > > > > those would work with the proposal in the KIP. > I > > > > don't > > > > > > know > > > > > > > > > > enough > > > > > > > > > > > > > about > > > > > > > > > > > > > > > each of the tools but seems like in some of the > > > > tools, > > > > > > you > > > > > > > > have > > > > > > > > > > to > > > > > > > > > > > > > define > > > > > > > > > > > > > > > some sort of class with parameter names as > config > > > > > names. > > > > > > How > > > > > > > > > will > > > > > > > > > > > > such > > > > > > > > > > > > > > > tools find out about the config values? In > Puppet, > > > if > > > > > > this > > > > > > > > > means > > > > > > > > > > > that > > > > > > > > > > > > > > each > > > > > > > > > > > > > > > Puppet agent has to read it from ZK, this > means the > > > > ZK > > > > > > port > > > > > > > > has > > > > > > > > > > to > > > > > > > > > > > be > > > > > > > > > > > > > > open > > > > > > > > > > > > > > > to pretty much every machine in the DC. This > is a > > > > > bummer > > > > > > and > > > > > > > > a > > > > > > > > > > very > > > > > > > > > > > > > > > confusing requirement. Not sure if this is > really a > > > > > > problem > > > > > > > > or > > > > > > > > > > not > > > > > > > > > > > > > (each > > > > > > > > > > > > > > of > > > > > > > > > > > > > > > those tools might behave differently), though > > > > pointing > > > > > > out > > > > > > > > that > > > > > > > > > > > this > > > > > > > > > > > > is > > > > > > > > > > > > > > > something worth paying attention to. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 3. The wrapper tools that let users read/change > > > > config > > > > > > tools > > > > > > > > > > should > > > > > > > > > > > > not > > > > > > > > > > > > > > > depend on ZK for the reason mentioned above. > It's a > > > > > pain > > > > > > to > > > > > > > > > > assume > > > > > > > > > > > > that > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > ZK port is open from any machine that needs to > run > > > > this > > > > > > tool. > > > > > > > > > > > Ideally > > > > > > > > > > > > > > what > > > > > > > > > > > > > > > users want is a REST API to the brokers to > change > > > or > > > > > > read the > > > > > > > > > > > config > > > > > > > > > > > > > (ala > > > > > > > > > > > > > > > Elasticsearch), but in the absence of the REST > API, > > > > we > > > > > > should > > > > > > > > > > think > > > > > > > > > > > > if > > > > > > > > > > > > > we > > > > > > > > > > > > > > > can write the tool such that it just requires > > > talking > > > > > to > > > > > > the > > > > > > > > > > Kafka > > > > > > > > > > > > > broker > > > > > > > > > > > > > > > port. This will require a config RPC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 4. Not sure if KIP is the right place to > discuss > > > the > > > > > > design > > > > > > > > of > > > > > > > > > > > > > > propagating > > > > > > > > > > > > > > > the config changes to the brokers, but have you > > > > thought > > > > > > about > > > > > > > > > > just > > > > > > > > > > > > > > letting > > > > > > > > > > > > > > > the controller oversee the config changes and > > > > propagate > > > > > > via > > > > > > > > RPC > > > > > > > > > > to > > > > > > > > > > > > the > > > > > > > > > > > > > > > brokers? That way, there is an easier way to > > > express > > > > > > config > > > > > > > > > > changes > > > > > > > > > > > > > that > > > > > > > > > > > > > > > require all brokers to change it for it to be > > > called > > > > > > > > complete. > > > > > > > > > > > Maybe > > > > > > > > > > > > > this > > > > > > > > > > > > > > > is not required, but it is hard to say if we > don't > > > > > > discuss > > > > > > > > the > > > > > > > > > > full > > > > > > > > > > > > set > > > > > > > > > > > > > > of > > > > > > > > > > > > > > > configs that need to be dynamic. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > Neha > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, May 1, 2015 at 12:53 PM, Jay Kreps < > > > > > > > > > jay.kr...@gmail.com <javascript:;>> > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hey Aditya, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This is a great! A couple of comments: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 1. Leaving the file config in place is > definitely > > > > the > > > > > > least > > > > > > > > > > > > > > disturbance. > > > > > > > > > > > > > > > > But let's really think about getting rid of > the > > > > files > > > > > > and > > > > > > > > > just > > > > > > > > > > > have > > > > > > > > > > > > > one > > > > > > > > > > > > > > > > config mechanism. There is always a tendency > to > > > > make > > > > > > > > > everything > > > > > > > > > > > > > > pluggable > > > > > > > > > > > > > > > > which so often just leads to two mediocre > > > > solutions. > > > > > > Can we > > > > > > > > > do > > > > > > > > > > > the > > > > > > > > > > > > > > exercise > > > > > > > > > > > > > > > > of trying to consider fully getting rid of > file > > > > > config > > > > > > and > > > > > > > > > > seeing > > > > > > > > > > > > > what > > > > > > > > > > > > > > goes > > > > > > > > > > > > > > > > wrong? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2. Do we need to model defaults? The current > > > > approach > > > > > > is > > > > > > > > that > > > > > > > > > > if > > > > > > > > > > > > you > > > > > > > > > > > > > > have a > > > > > > > > > > > > > > > > global config x it is overridden for a topic > xyz > > > by > > > > > > > > > > > /topics/xyz/x, > > > > > > > > > > > > > and > > > > > > > > > > > > > > I > > > > > > > > > > > > > > > > think this could be extended to > /brokers/0/x. I > > > > think > > > > > > this > > > > > > > > is > > > > > > > > > > > > > simpler. > > > > > > > > > > > > > > We > > > > > > > > > > > > > > > > need to specify the precedence for these > > > overrides, > > > > > > e.g. if > > > > > > > > > you > > > > > > > > > > > > > > override at > > > > > > > > > > > > > > > > the broker and topic level I think the topic > > > level > > > > > > takes > > > > > > > > > > > > precedence. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 3. I recommend we have the producer and > consumer > > > > > config > > > > > > > > just > > > > > > > > > be > > > > > > > > > > > an > > > > > > > > > > > > > > override > > > > > > > > > > > > > > > > under client.id. The override is by client > id > > > and > > > > we > > > > > > can > > > > > > > > > have > > > > > > > > > > > > > separate > > > > > > > > > > > > > > > > properties for controlling quotas for > producers > > > and > > > > > > > > > consumers. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 4. Some configs can be changed just by > updating > > > the > > > > > > > > > reference, > > > > > > > > > > > > others > > > > > > > > > > > > > > may > > > > > > > > > > > > > > > > require some action. An example of this is > if you > > > > > want > > > > > > to > > > > > > > > > > disable > > > > > > > > > > > > log > > > > > > > > > > > > > > > > compaction (assuming we wanted to make that > > > > dynamic) > > > > > we > > > > > > > > need > > > > > > > > > to > > > > > > > > > > > > call > > > > > > > > > > > > > > > > shutdown() on the cleaner. I think it may be > > > > required > > > > > > to > > > > > > > > > > > register a > > > > > > > > > > > > > > > > listener callback that gets called when the > > > config > > > > > > changes. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 5. For handling the reference can you explain > > > your > > > > > > plan a > > > > > > > > > bit? > > > > > > > > > > > > > > Currently we > > > > > > > > > > > > > > > > have an immutable KafkaConfig object with a > bunch > > > > of > > > > > > vals. > > > > > > > > > That > > > > > > > > > > > or > > > > > > > > > > > > > > > > individual values in there get injected all > over > > > > the > > > > > > code > > > > > > > > > > base. I > > > > > > > > > > > > was > > > > > > > > > > > > > > > > thinking something like this: > > > > > > > > > > > > > > > > a. We retain the KafkaConfig object as an > > > immutable > > > > > > object > > > > > > > > > just > > > > > > > > > > > as > > > > > > > > > > > > > > today. > > > > > > > > > > > > > > > > b. It is no longer legit to grab values out > fo > > > that > > > > > > config > > > > > > > > if > > > > > > > > > > > they > > > > > > > > > > > > > are > > > > > > > > > > > > > > > > changeable. > > > > > > > > > > > > > > > > c. Instead of making KafkaConfig itself > mutable > > > we > > > > > make > > > > > > > > > > > > > > KafkaConfiguration > > > > > > > > > > > > > > > > which has a single volatile reference to the > > > > current > > > > > > > > > > KafkaConfig. > > > > > > > > > > > > > > > > KafkaConfiguration is what gets passed into > > > various > > > > > > > > > components. > > > > > > > > > > > So > > > > > > > > > > > > to > > > > > > > > > > > > > > > > access a config you do something like > > > > > > > > > config.instance.myValue. > > > > > > > > > > > When > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > config changes the config manager updates > this > > > > > > reference. > > > > > > > > > > > > > > > > d. The KafkaConfiguration is the thing that > > > allows > > > > > > doing > > > > > > > > the > > > > > > > > > > > > > > > > configuration.onChange("my.config", callback) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -Jay > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Apr 28, 2015 at 3:57 PM, Aditya > Auradkar > > > < > > > > > > > > > > > > > > > > aaurad...@linkedin.com.invalid> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hey everyone, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Wrote up a KIP to update topic, client and > > > broker > > > > > > configs > > > > > > > > > > > > > > dynamically via > > > > > > > > > > > > > > > > > Zookeeper. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please read and provide feedback. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > Aditya > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > PS: I've intentionally kept this discussion > > > > > separate > > > > > > from > > > > > > > > > > KIP-5 > > > > > > > > > > > > > > since I'm > > > > > > > > > > > > > > > > > not sure if that is actively being worked > on > > > and > > > > I > > > > > > wanted > > > > > > > > > to > > > > > > > > > > > > start > > > > > > > > > > > > > > with a > > > > > > > > > > > > > > > > > clean slate. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > Neha > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > Joel > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Ashish ?h > > > > > > > > > > > > > > > -- > > Thanks, > > Neha >
