Updated KIP regarding how broker JSON version will be handled and new procedure of upgrade.
On Wed, Jan 13, 2016 at 10:58 AM, Allen Wang <allenxw...@gmail.com> wrote: > Hi Jun, > > I feel it is a bit complicated and unconventional to have a major release > have dependency on a minor release. It would also double the cost of > releasing. > > On the other hand, if we want to skip the minor release and fix > ZkUtils.getBrokerInfo() in 0.9.1, it would require all 0.9.0 clients to > upgrade first to 0.9.1 before the upgrading brokers, which is also a bit > unconventional. It is also problematic since 0.9.1 client will expect rack > field (even though it may be encoded as null) when it tries to deserialize > TopicMetadataResponse. > > So it looks like we have to have a minor release first. > > Thanks, > Allen > > > > On Tue, Jan 12, 2016 at 6:27 PM, Jun Rao <j...@confluent.io> wrote: > >> Allen, >> >> It's not ideal to add a new field in json without increasing the version. >> Also, if we don't fix this issue in 0.9.0, if we ever change the version >> of >> json in the future, the consumer in 0.9.0 will break after the broker is >> upgraded to the new release. So, I suggest that we fix the behavior in >> ZkUtils.getBrokerInfo() >> in both trunk and 0.9.0 branch. After we release 0.9.0.1, the upgrade path >> is for the old consumer to be upgraded to 0.9.0.1 before upgrading the >> broker to 0.9.1 and beyond. This fix can be done in a separate jira. >> >> Thanks, >> >> Jun >> >> On Tue, Jan 12, 2016 at 5:35 PM, Allen Wang <allenxw...@gmail.com> wrote: >> >> > Agreed. So it seems that for 0.9.1, the only option is to keep the JSON >> > version unchanged. But as part of the PR, I can change the behavior of >> > ZkUtils.getBrokerInfo() >> > to make it compatible with future JSON versions. >> > >> > Thanks, >> > Allen >> > >> > >> > On Tue, Jan 12, 2016 at 2:57 PM, Jun Rao <j...@confluent.io> wrote: >> > >> > > Hi, Allen, >> > > >> > > That's a good point. In 0.9.0.0, the old consumer reads broker info >> > > directly from ZK and the code throws an exception if the version in >> json >> > is >> > > not 1 or 2. This old consumer will break when we upgrade the broker >> json >> > to >> > > version 3 in ZK in 0.9.1, which will be an issue. We overlooked this >> > issue >> > > in 0.9.0.0. The easiest fix is probably not to check the version in >> > > ZkUtils.getBrokerInfo(). >> > > This way, as long as we are only adding new fields in broker json, we >> can >> > > preserve the compatibility. >> > > >> > > Thanks, >> > > >> > > Jun >> > > >> > > On Tue, Jan 12, 2016 at 1:52 PM, Allen Wang <allenxw...@gmail.com> >> > wrote: >> > > >> > > > Hi Jun, >> > > > >> > > > That's a good suggestion. However, it does not solve the problem for >> > the >> > > > clients or thirty party tools that get broker information directly >> from >> > > > ZooKeeper. >> > > > >> > > > Thanks, >> > > > Allen >> > > > >> > > > >> > > > On Tue, Jan 12, 2016 at 1:29 PM, Jun Rao <j...@confluent.io> wrote: >> > > > >> > > > > Allen, >> > > > > >> > > > > Another way to do this is the following. >> > > > > >> > > > > When inter.broker.protocol.version is set to 0.9.0, the broker >> will >> > > write >> > > > > the broker info in ZK using version 2, ignoring the rack info. >> > > > > >> > > > > When inter.broker.protocol.version is set to 0.9.1, the broker >> will >> > > write >> > > > > the broker info in ZK using version 3, including the rack info. >> > > > > >> > > > > If one follows the upgrade process, after the 2nd round of rolling >> > > > bounces, >> > > > > every broker is capable of parsing version 3 of broker info in ZK. >> > This >> > > > is >> > > > > when the rack-aware feature will be used. >> > > > > >> > > > > >> > > > > Thanks, >> > > > > >> > > > > Jun >> > > > > >> > > > > On Tue, Jan 12, 2016 at 12:19 PM, Allen Wang < >> allenxw...@gmail.com> >> > > > wrote: >> > > > > >> > > > > > Regarding the JSON version of Broker: >> > > > > > >> > > > > > I don't why the ZkUtils.getBrokerInfo() restricts the JSON >> versions >> > > it >> > > > > can >> > > > > > read. It will throw exception if version is not 1 or 2. Seems >> to me >> > > > that >> > > > > it >> > > > > > will cause compatibility problem whenever the version needs to >> be >> > > > changed >> > > > > > and make the upgrade path difficult. >> > > > > > >> > > > > > One option we have is to make rack also part of version 2 and >> keep >> > > the >> > > > > > version 2 unchanged for this update. This will make the old >> clients >> > > > > > compatible. During rolling upgrade, it will also avoid problems >> if >> > > the >> > > > > > controller/broker is still the old version. >> > > > > > >> > > > > > However, ZkUtils.getBrokerInfo() will be updated to return the >> > Broker >> > > > > with >> > > > > > rack so the rack information will be available once the >> > server/client >> > > > is >> > > > > > upgraded to the latest version. >> > > > > > >> > > > > > >> > > > > > >> > > > > > On Wed, Jan 6, 2016 at 6:28 PM, Allen Wang < >> allenxw...@gmail.com> >> > > > wrote: >> > > > > > >> > > > > > > Updated KIP according to Jun's comment and included changes to >> > TMR. >> > > > > > > >> > > > > > > On Tue, Jan 5, 2016 at 5:59 PM, Jun Rao <j...@confluent.io> >> > wrote: >> > > > > > > >> > > > > > >> Hi, Allen, >> > > > > > >> >> > > > > > >> A couple of minor comments on the KIP. >> > > > > > >> >> > > > > > >> 1. The version of the broker JSON string says 2. It should >> be 3. >> > > > > > >> >> > > > > > >> 2. The new version of UpdateMetadataRequest should be 2, >> instead >> > > of >> > > > 1. >> > > > > > >> Could you include the full wire protocol of version 2 of >> > > > > > >> UpdateMetadataRequest and highlight the changed part? >> > > > > > >> >> > > > > > >> Thanks, >> > > > > > >> >> > > > > > >> Jun >> > > > > > >> >> > > > > > >> On Tue, Jan 5, 2016 at 3:11 PM, Allen Wang < >> > allenxw...@gmail.com> >> > > > > > wrote: >> > > > > > >> >> > > > > > >> > Jun and I had a chance to discuss it in a meeting and it is >> > > agreed >> > > > > to >> > > > > > >> > change the TMR in a different patch. >> > > > > > >> > >> > > > > > >> > I can change the KIP to include rack in TMR. The essential >> > > change >> > > > is >> > > > > > to >> > > > > > >> add >> > > > > > >> > rack into class BrokerEndPoint and make TMR version aware. >> > > > > > >> > >> > > > > > >> > >> > > > > > >> > >> > > > > > >> > On Tue, Jan 5, 2016 at 10:21 AM, Aditya Auradkar < >> > > > > > >> > aaurad...@linkedin.com.invalid> wrote: >> > > > > > >> > >> > > > > > >> > > Jun/Allen - >> > > > > > >> > > >> > > > > > >> > > Did we ever actually agree on whether we should evolve >> the >> > TMR >> > > > to >> > > > > > >> include >> > > > > > >> > > rack info or not? >> > > > > > >> > > I don't feel strongly about it but I if it's the right >> thing >> > > to >> > > > do >> > > > > > we >> > > > > > >> > > should probably do it in this KIP (can be a separate >> > patch).. >> > > it >> > > > > > >> isn't a >> > > > > > >> > > large change. >> > > > > > >> > > >> > > > > > >> > > Aditya >> > > > > > >> > > >> > > > > > >> > > On Sat, Dec 26, 2015 at 3:01 PM, Allen Wang < >> > > > allenxw...@gmail.com >> > > > > > >> > > > > > >> > wrote: >> > > > > > >> > > >> > > > > > >> > > > Added the rolling upgrade instruction in the KIP, >> similar >> > to >> > > > > those >> > > > > > >> in >> > > > > > >> > > 0.9.0 >> > > > > > >> > > > release notes. >> > > > > > >> > > > >> > > > > > >> > > > On Wed, Dec 16, 2015 at 11:32 AM, Allen Wang < >> > > > > > allenxw...@gmail.com> >> > > > > > >> > > wrote: >> > > > > > >> > > > >> > > > > > >> > > > > Hi Jun, >> > > > > > >> > > > > >> > > > > > >> > > > > The reason that TopicMetadataResponse is not >> included in >> > > the >> > > > > KIP >> > > > > > >> is >> > > > > > >> > > that >> > > > > > >> > > > > it currently is not version aware . So we need to >> > > introduce >> > > > > > >> version >> > > > > > >> > to >> > > > > > >> > > it >> > > > > > >> > > > > in order to make sure backward compatibility. It >> seems >> > to >> > > > me a >> > > > > > big >> > > > > > >> > > > change. >> > > > > > >> > > > > Do we want to couple it with this KIP? Do we need to >> > > further >> > > > > > >> discuss >> > > > > > >> > > what >> > > > > > >> > > > > information to include in the new version besides >> rack? >> > > For >> > > > > > >> example, >> > > > > > >> > > > should >> > > > > > >> > > > > we include broker security protocol in >> > > > TopicMetadataResponse? >> > > > > > >> > > > > >> > > > > > >> > > > > The other option is to make it a separate KIP to make >> > > > > > >> > > > > TopicMetadataResponse version aware and decide what >> to >> > > > > include, >> > > > > > >> and >> > > > > > >> > > make >> > > > > > >> > > > > this KIP focus on the rack aware algorithm, admin >> tools >> > > and >> > > > > > >> related >> > > > > > >> > > > > changes to inter-broker protocol . >> > > > > > >> > > > > >> > > > > > >> > > > > Thanks, >> > > > > > >> > > > > Allen >> > > > > > >> > > > > >> > > > > > >> > > > > >> > > > > > >> > > > > >> > > > > > >> > > > > >> > > > > > >> > > > > On Mon, Dec 14, 2015 at 8:30 AM, Jun Rao < >> > > j...@confluent.io> >> > > > > > >> wrote: >> > > > > > >> > > > > >> > > > > > >> > > > >> Allen, >> > > > > > >> > > > >> >> > > > > > >> > > > >> Thanks for the proposal. A few comments. >> > > > > > >> > > > >> >> > > > > > >> > > > >> 1. Since this KIP changes the inter broker >> > communication >> > > > > > protocol >> > > > > > >> > > > >> (UpdateMetadataRequest), we will need to document >> the >> > > > upgrade >> > > > > > >> path >> > > > > > >> > > > >> (similar >> > > > > > >> > > > >> to what's described in >> > > > > > >> > > > >> >> http://kafka.apache.org/090/documentation.html#upgrade >> > ). >> > > > > > >> > > > >> >> > > > > > >> > > > >> 2. It might be useful to include the rack info of >> the >> > > > broker >> > > > > in >> > > > > > >> > > > >> TopicMetadataResponse. This can be useful for >> > > > administrative >> > > > > > >> tasks, >> > > > > > >> > as >> > > > > > >> > > > >> well >> > > > > > >> > > > >> as read affinity in the future. >> > > > > > >> > > > >> >> > > > > > >> > > > >> Jun >> > > > > > >> > > > >> >> > > > > > >> > > > >> >> > > > > > >> > > > >> >> > > > > > >> > > > >> On Thu, Dec 10, 2015 at 9:38 AM, Allen Wang < >> > > > > > >> allenxw...@gmail.com> >> > > > > > >> > > > wrote: >> > > > > > >> > > > >> >> > > > > > >> > > > >> > If there are no more comments I would like to call >> > for >> > > a >> > > > > > vote. >> > > > > > >> > > > >> > >> > > > > > >> > > > >> > >> > > > > > >> > > > >> > On Sun, Nov 15, 2015 at 10:08 PM, Allen Wang < >> > > > > > >> > allenxw...@gmail.com> >> > > > > > >> > > > >> wrote: >> > > > > > >> > > > >> > >> > > > > > >> > > > >> > > KIP is updated with more details and how to >> handle >> > > the >> > > > > > >> situation >> > > > > > >> > > > where >> > > > > > >> > > > >> > > rack information is incomplete. >> > > > > > >> > > > >> > > >> > > > > > >> > > > >> > > In the situation where rack information is >> > > incomplete, >> > > > > but >> > > > > > we >> > > > > > >> > want >> > > > > > >> > > > to >> > > > > > >> > > > >> > > continue with the assignment, I have suggested >> to >> > > > ignore >> > > > > > all >> > > > > > >> > rack >> > > > > > >> > > > >> > > information and fallback to original algorithm. >> The >> > > > > reason >> > > > > > is >> > > > > > >> > > > >> explained >> > > > > > >> > > > >> > > below: >> > > > > > >> > > > >> > > >> > > > > > >> > > > >> > > The other options are to assume that the broker >> > > without >> > > > > the >> > > > > > >> rack >> > > > > > >> > > > >> belong >> > > > > > >> > > > >> > to >> > > > > > >> > > > >> > > its own unique rack, or they belong to one >> > "default" >> > > > > rack. >> > > > > > >> > Either >> > > > > > >> > > > way >> > > > > > >> > > > >> we >> > > > > > >> > > > >> > > choose, it is highly likely to result in uneven >> > > number >> > > > of >> > > > > > >> > brokers >> > > > > > >> > > in >> > > > > > >> > > > >> > racks, >> > > > > > >> > > > >> > > and it is quite possible that the "made up" >> racks >> > > will >> > > > > have >> > > > > > >> much >> > > > > > >> > > > fewer >> > > > > > >> > > > >> > > number of brokers. As I explained in the KIP, >> > uneven >> > > > > number >> > > > > > >> of >> > > > > > >> > > > >> brokers in >> > > > > > >> > > > >> > > racks will lead to uneven distribution of >> replicas >> > > > among >> > > > > > >> brokers >> > > > > > >> > > > (even >> > > > > > >> > > > >> > > though the leader distribution is still even). >> The >> > > > > brokers >> > > > > > in >> > > > > > >> > the >> > > > > > >> > > > rack >> > > > > > >> > > > >> > that >> > > > > > >> > > > >> > > has fewer number of brokers will get more >> replicas >> > > per >> > > > > > broker >> > > > > > >> > than >> > > > > > >> > > > >> > brokers >> > > > > > >> > > > >> > > in other racks. >> > > > > > >> > > > >> > > >> > > > > > >> > > > >> > > Given this fact and the replica assignment >> produced >> > > > will >> > > > > be >> > > > > > >> > > > incorrect >> > > > > > >> > > > >> > > anyway from rack aware point of view, ignoring >> all >> > > rack >> > > > > > >> > > information >> > > > > > >> > > > >> and >> > > > > > >> > > > >> > > fallback to the original algorithm is not a bad >> > > choice >> > > > > > since >> > > > > > >> it >> > > > > > >> > > will >> > > > > > >> > > > >> at >> > > > > > >> > > > >> > > least have a better guarantee of replica >> > > distribution. >> > > > > > >> > > > >> > > >> > > > > > >> > > > >> > > Also for command line tools it gives user a >> choice >> > if >> > > > for >> > > > > > any >> > > > > > >> > > reason >> > > > > > >> > > > >> they >> > > > > > >> > > > >> > > want to ignore rack information and fallback to >> the >> > > > > > original >> > > > > > >> > > > >> algorithm. >> > > > > > >> > > > >> > > >> > > > > > >> > > > >> > > >> > > > > > >> > > > >> > > On Tue, Nov 10, 2015 at 9:04 AM, Allen Wang < >> > > > > > >> > allenxw...@gmail.com >> > > > > > >> > > > >> > > > > > >> > > > >> > wrote: >> > > > > > >> > > > >> > > >> > > > > > >> > > > >> > >> I am busy with some time pressing issues for >> the >> > > last >> > > > > few >> > > > > > >> > days. I >> > > > > > >> > > > >> will >> > > > > > >> > > > >> > >> think about how the incomplete rack information >> > will >> > > > > > affect >> > > > > > >> the >> > > > > > >> > > > >> balance >> > > > > > >> > > > >> > and >> > > > > > >> > > > >> > >> update the KIP by early next week. >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> > >> Thanks, >> > > > > > >> > > > >> > >> Allen >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> > >> On Tue, Nov 3, 2015 at 9:03 AM, Neha Narkhede < >> > > > > > >> > n...@confluent.io >> > > > > > >> > > > >> > > > > > >> > > > >> > wrote: >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> > >>> Few suggestions on improving the KIP >> > > > > > >> > > > >> > >>> >> > > > > > >> > > > >> > >>> *If some brokers have rack, and some do not, >> the >> > > > > > algorithm >> > > > > > >> > will >> > > > > > >> > > > >> thrown >> > > > > > >> > > > >> > an >> > > > > > >> > > > >> > >>> > exception. This is to prevent incorrect >> > > assignment >> > > > > > >> caused by >> > > > > > >> > > > user >> > > > > > >> > > > >> > >>> error.* >> > > > > > >> > > > >> > >>> >> > > > > > >> > > > >> > >>> >> > > > > > >> > > > >> > >>> In the KIP, can you clearly state the >> user-facing >> > > > > > behavior >> > > > > > >> > when >> > > > > > >> > > > some >> > > > > > >> > > > >> > >>> brokers have rack information and some don't. >> > Which >> > > > > > actions >> > > > > > >> > and >> > > > > > >> > > > >> > requests >> > > > > > >> > > > >> > >>> will error out and how? >> > > > > > >> > > > >> > >>> >> > > > > > >> > > > >> > >>> *Even distribution of partition leadership >> among >> > > > > brokers* >> > > > > > >> > > > >> > >>> >> > > > > > >> > > > >> > >>> >> > > > > > >> > > > >> > >>> There is some information about arranging the >> > > sorted >> > > > > > broker >> > > > > > >> > list >> > > > > > >> > > > >> > >>> interlaced >> > > > > > >> > > > >> > >>> with rack ids. Can you describe the changes to >> > the >> > > > > > current >> > > > > > >> > > > algorithm >> > > > > > >> > > > >> > in a >> > > > > > >> > > > >> > >>> little more detail? How does this interlacing >> > work >> > > if >> > > > > > only >> > > > > > >> a >> > > > > > >> > > > subset >> > > > > > >> > > > >> of >> > > > > > >> > > > >> > >>> brokers have the rack id configured? Does this >> > > still >> > > > > work >> > > > > > >> if >> > > > > > >> > > > uneven >> > > > > > >> > > > >> # >> > > > > > >> > > > >> > of >> > > > > > >> > > > >> > >>> brokers are assigned to each rack? It might >> work, >> > > I'm >> > > > > > >> looking >> > > > > > >> > > for >> > > > > > >> > > > >> more >> > > > > > >> > > > >> > >>> details on the changes, since it will affect >> the >> > > > > behavior >> > > > > > >> seen >> > > > > > >> > > by >> > > > > > >> > > > >> the >> > > > > > >> > > > >> > >>> user >> > > > > > >> > > > >> > >>> - imbalance on either the leaders or data or >> > both. >> > > > > > >> > > > >> > >>> >> > > > > > >> > > > >> > >>> On Mon, Nov 2, 2015 at 6:39 PM, Aditya >> Auradkar < >> > > > > > >> > > > >> > aaurad...@linkedin.com> >> > > > > > >> > > > >> > >>> wrote: >> > > > > > >> > > > >> > >>> >> > > > > > >> > > > >> > >>> > I think this sounds reasonable. Anyone else >> > have >> > > > > > >> comments? >> > > > > > >> > > > >> > >>> > >> > > > > > >> > > > >> > >>> > Aditya >> > > > > > >> > > > >> > >>> > >> > > > > > >> > > > >> > >>> > On Tue, Oct 27, 2015 at 5:23 PM, Allen Wang >> < >> > > > > > >> > > > allenxw...@gmail.com >> > > > > > >> > > > >> > >> > > > > > >> > > > >> > >>> wrote: >> > > > > > >> > > > >> > >>> > >> > > > > > >> > > > >> > >>> > > During the discussion in the hangout, it >> was >> > > > > > mentioned >> > > > > > >> > that >> > > > > > >> > > it >> > > > > > >> > > > >> > would >> > > > > > >> > > > >> > >>> be >> > > > > > >> > > > >> > >>> > > desirable that consumers know the rack >> > > > information >> > > > > of >> > > > > > >> the >> > > > > > >> > > > >> brokers >> > > > > > >> > > > >> > so >> > > > > > >> > > > >> > >>> that >> > > > > > >> > > > >> > >>> > > they can consume from the broker in the >> same >> > > rack >> > > > > to >> > > > > > >> > reduce >> > > > > > >> > > > >> > latency. >> > > > > > >> > > > >> > >>> As I >> > > > > > >> > > > >> > >>> > > understand this will only be beneficial if >> > > > consumer >> > > > > > can >> > > > > > >> > > > consume >> > > > > > >> > > > >> > from >> > > > > > >> > > > >> > >>> any >> > > > > > >> > > > >> > >>> > > broker in ISR, which is not possible now. >> > > > > > >> > > > >> > >>> > > >> > > > > > >> > > > >> > >>> > > I suggest we skip the change to TMR. Once >> the >> > > > > change >> > > > > > is >> > > > > > >> > made >> > > > > > >> > > > to >> > > > > > >> > > > >> > >>> consumer >> > > > > > >> > > > >> > >>> > to >> > > > > > >> > > > >> > >>> > > be able to consume from any broker in ISR, >> > the >> > > > rack >> > > > > > >> > > > information >> > > > > > >> > > > >> can >> > > > > > >> > > > >> > >>> be >> > > > > > >> > > > >> > >>> > > added to TMR. >> > > > > > >> > > > >> > >>> > > >> > > > > > >> > > > >> > >>> > > Another thing I want to confirm is >> command >> > > line >> > > > > > >> > behavior. I >> > > > > > >> > > > >> think >> > > > > > >> > > > >> > >>> the >> > > > > > >> > > > >> > >>> > > desirable default behavior is to fail >> fast on >> > > > > command >> > > > > > >> line >> > > > > > >> > > for >> > > > > > >> > > > >> > >>> incomplete >> > > > > > >> > > > >> > >>> > > rack mapping. The error message can >> include >> > > > further >> > > > > > >> > > > instruction >> > > > > > >> > > > >> > that >> > > > > > >> > > > >> > >>> > tells >> > > > > > >> > > > >> > >>> > > the user to add an extra argument (like >> > > > > > >> > > > >> "--allow-partial-rackinfo") >> > > > > > >> > > > >> > >>> to >> > > > > > >> > > > >> > >>> > > suppress the error and do an imperfect >> rack >> > > aware >> > > > > > >> > > assignment. >> > > > > > >> > > > If >> > > > > > >> > > > >> > the >> > > > > > >> > > > >> > >>> > > default behavior is to allow incomplete >> > > mapping, >> > > > > the >> > > > > > >> error >> > > > > > >> > > can >> > > > > > >> > > > >> > still >> > > > > > >> > > > >> > >>> be >> > > > > > >> > > > >> > >>> > > easily missed. >> > > > > > >> > > > >> > >>> > > >> > > > > > >> > > > >> > >>> > > The affected command line tools are >> > > TopicCommand >> > > > > and >> > > > > > >> > > > >> > >>> > > ReassignPartitionsCommand. >> > > > > > >> > > > >> > >>> > > >> > > > > > >> > > > >> > >>> > > Thanks, >> > > > > > >> > > > >> > >>> > > Allen >> > > > > > >> > > > >> > >>> > > >> > > > > > >> > > > >> > >>> > > >> > > > > > >> > > > >> > >>> > > >> > > > > > >> > > > >> > >>> > > >> > > > > > >> > > > >> > >>> > > >> > > > > > >> > > > >> > >>> > > On Mon, Oct 26, 2015 at 12:55 PM, Aditya >> > > > Auradkar < >> > > > > > >> > > > >> > >>> > aaurad...@linkedin.com> >> > > > > > >> > > > >> > >>> > > wrote: >> > > > > > >> > > > >> > >>> > > >> > > > > > >> > > > >> > >>> > > > Hi Allen, >> > > > > > >> > > > >> > >>> > > > >> > > > > > >> > > > >> > >>> > > > For TopicMetadataResponse to understand >> > > > version, >> > > > > > you >> > > > > > >> can >> > > > > > >> > > > bump >> > > > > > >> > > > >> up >> > > > > > >> > > > >> > >>> the >> > > > > > >> > > > >> > >>> > > > request version itself. Based on the >> > version >> > > of >> > > > > the >> > > > > > >> > > request, >> > > > > > >> > > > >> the >> > > > > > >> > > > >> > >>> > response >> > > > > > >> > > > >> > >>> > > > can be appropriately serialized. It >> > shouldn't >> > > > be >> > > > > a >> > > > > > >> huge >> > > > > > >> > > > >> change. >> > > > > > >> > > > >> > For >> > > > > > >> > > > >> > >>> > > > example: We went through something >> similar >> > > for >> > > > > > >> > > > ProduceRequest >> > > > > > >> > > > >> > >>> recently >> > > > > > >> > > > >> > >>> > ( >> > > > > > >> > > > >> > >>> > > > https://reviews.apache.org/r/33378/) >> > > > > > >> > > > >> > >>> > > > I guess the reason protocol information >> is >> > > not >> > > > > > >> included >> > > > > > >> > in >> > > > > > >> > > > the >> > > > > > >> > > > >> > TMR >> > > > > > >> > > > >> > >>> is >> > > > > > >> > > > >> > >>> > > > because the topic itself is independent >> of >> > > any >> > > > > > >> > particular >> > > > > > >> > > > >> > protocol >> > > > > > >> > > > >> > >>> (SSL >> > > > > > >> > > > >> > >>> > > vs >> > > > > > >> > > > >> > >>> > > > Plaintext). Having said that, I'm not >> sure >> > we >> > > > > even >> > > > > > >> need >> > > > > > >> > > rack >> > > > > > >> > > > >> > >>> > information >> > > > > > >> > > > >> > >>> > > in >> > > > > > >> > > > >> > >>> > > > TMR. What usecase were you thinking of >> > > > initially? >> > > > > > >> > > > >> > >>> > > > >> > > > > > >> > > > >> > >>> > > > For 1 - I'd be fine with adding an >> option >> > to >> > > > the >> > > > > > >> command >> > > > > > >> > > > line >> > > > > > >> > > > >> > tools >> > > > > > >> > > > >> > >>> > that >> > > > > > >> > > > >> > >>> > > > check rack assignment. For e.g. >> > > > > > >> "--strict-assignment" or >> > > > > > >> > > > >> > something >> > > > > > >> > > > >> > >>> > > similar. >> > > > > > >> > > > >> > >>> > > > >> > > > > > >> > > > >> > >>> > > > Aditya >> > > > > > >> > > > >> > >>> > > > >> > > > > > >> > > > >> > >>> > > > On Thu, Oct 22, 2015 at 6:44 PM, Allen >> > Wang < >> > > > > > >> > > > >> > allenxw...@gmail.com> >> > > > > > >> > > > >> > >>> > > wrote: >> > > > > > >> > > > >> > >>> > > > >> > > > > > >> > > > >> > >>> > > > > For 2 and 3, I have updated the KIP. >> > Please >> > > > > take >> > > > > > a >> > > > > > >> > look. >> > > > > > >> > > > One >> > > > > > >> > > > >> > >>> thing I >> > > > > > >> > > > >> > >>> > > have >> > > > > > >> > > > >> > >>> > > > > changed is removing the proposal to >> add >> > > rack >> > > > to >> > > > > > >> > > > >> > >>> > TopicMetadataResponse. >> > > > > > >> > > > >> > >>> > > > The >> > > > > > >> > > > >> > >>> > > > > reason is that unlike >> > > UpdateMetadataRequest, >> > > > > > >> > > > >> > >>> TopicMetadataResponse >> > > > > > >> > > > >> > >>> > does >> > > > > > >> > > > >> > >>> > > > not >> > > > > > >> > > > >> > >>> > > > > understand version. I don't see a way >> to >> > > > > include >> > > > > > >> rack >> > > > > > >> > > > >> without >> > > > > > >> > > > >> > >>> > breaking >> > > > > > >> > > > >> > >>> > > > old >> > > > > > >> > > > >> > >>> > > > > version of clients. That's probably >> why >> > > > secure >> > > > > > >> > protocol >> > > > > > >> > > is >> > > > > > >> > > > >> not >> > > > > > >> > > > >> > >>> > included >> > > > > > >> > > > >> > >>> > > > in >> > > > > > >> > > > >> > >>> > > > > the TopicMetadataResponse either. I >> think >> > > it >> > > > > will >> > > > > > >> be a >> > > > > > >> > > > much >> > > > > > >> > > > >> > >>> bigger >> > > > > > >> > > > >> > >>> > > change >> > > > > > >> > > > >> > >>> > > > > to include rack in >> TopicMetadataResponse. >> > > > > > >> > > > >> > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > For 1, my concern is that doing rack >> > aware >> > > > > > >> assignment >> > > > > > >> > > > >> without >> > > > > > >> > > > >> > >>> > complete >> > > > > > >> > > > >> > >>> > > > > broker to rack mapping will result in >> > > > > assignment >> > > > > > >> that >> > > > > > >> > is >> > > > > > >> > > > not >> > > > > > >> > > > >> > rack >> > > > > > >> > > > >> > >>> > aware >> > > > > > >> > > > >> > >>> > > > and >> > > > > > >> > > > >> > >>> > > > > fail to provide fault tolerance in the >> > > event >> > > > of >> > > > > > >> rack >> > > > > > >> > > > outage. >> > > > > > >> > > > >> > This >> > > > > > >> > > > >> > >>> > kind >> > > > > > >> > > > >> > >>> > > of >> > > > > > >> > > > >> > >>> > > > > problem will be difficult to surface. >> And >> > > the >> > > > > > cost >> > > > > > >> of >> > > > > > >> > > this >> > > > > > >> > > > >> > >>> problem is >> > > > > > >> > > > >> > >>> > > > high: >> > > > > > >> > > > >> > >>> > > > > you have to do partition reassignment >> if >> > > you >> > > > > are >> > > > > > >> lucky >> > > > > > >> > > to >> > > > > > >> > > > >> spot >> > > > > > >> > > > >> > >>> the >> > > > > > >> > > > >> > >>> > > > problem >> > > > > > >> > > > >> > >>> > > > > early on or face the consequence of >> data >> > > loss >> > > > > > >> during >> > > > > > >> > > real >> > > > > > >> > > > >> rack >> > > > > > >> > > > >> > >>> > outage. >> > > > > > >> > > > >> > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > I do see the concern of fail-fast as >> it >> > > might >> > > > > > also >> > > > > > >> > cause >> > > > > > >> > > > >> data >> > > > > > >> > > > >> > >>> loss if >> > > > > > >> > > > >> > >>> > > > > producer is not able produce the >> message >> > > due >> > > > to >> > > > > > >> topic >> > > > > > >> > > > >> creation >> > > > > > >> > > > >> > >>> > failure. >> > > > > > >> > > > >> > >>> > > > Is >> > > > > > >> > > > >> > >>> > > > > it feasible to treat dynamic topic >> > creation >> > > > and >> > > > > > >> > command >> > > > > > >> > > > >> tools >> > > > > > >> > > > >> > >>> > > > differently? >> > > > > > >> > > > >> > >>> > > > > We allow dynamic topic creation with >> > > > incomplete >> > > > > > >> > > > broker-rack >> > > > > > >> > > > >> > >>> mapping >> > > > > > >> > > > >> > >>> > and >> > > > > > >> > > > >> > >>> > > > > fail fast in command line. Another >> option >> > > is >> > > > to >> > > > > > let >> > > > > > >> > user >> > > > > > >> > > > >> > >>> determine >> > > > > > >> > > > >> > >>> > the >> > > > > > >> > > > >> > >>> > > > > behavior for command line. For >> example, >> > by >> > > > > > default >> > > > > > >> > fail >> > > > > > >> > > > >> fast in >> > > > > > >> > > > >> > >>> > command >> > > > > > >> > > > >> > >>> > > > > line but allow incomplete broker-rack >> > > mapping >> > > > > if >> > > > > > >> > another >> > > > > > >> > > > >> switch >> > > > > > >> > > > >> > >>> is >> > > > > > >> > > > >> > >>> > > > > provided. >> > > > > > >> > > > >> > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > On Tue, Oct 20, 2015 at 10:05 AM, >> Aditya >> > > > > > Auradkar < >> > > > > > >> > > > >> > >>> > > > > aaurad...@linkedin.com.invalid> >> wrote: >> > > > > > >> > > > >> > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > > Hey Allen, >> > > > > > >> > > > >> > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > > 1. If we choose fail fast topic >> > creation, >> > > > we >> > > > > > will >> > > > > > >> > have >> > > > > > >> > > > >> topic >> > > > > > >> > > > >> > >>> > creation >> > > > > > >> > > > >> > >>> > > > > > failures while upgrading the >> cluster. I >> > > > > really >> > > > > > >> doubt >> > > > > > >> > > we >> > > > > > >> > > > >> want >> > > > > > >> > > > >> > >>> this >> > > > > > >> > > > >> > >>> > > > > behavior. >> > > > > > >> > > > >> > >>> > > > > > Ideally, this should be invisible to >> > > > clients >> > > > > > of a >> > > > > > >> > > > cluster. >> > > > > > >> > > > >> > >>> > Currently, >> > > > > > >> > > > >> > >>> > > > > each >> > > > > > >> > > > >> > >>> > > > > > broker is effectively its own rack. >> So >> > we >> > > > > > >> probably >> > > > > > >> > can >> > > > > > >> > > > use >> > > > > > >> > > > >> > the >> > > > > > >> > > > >> > >>> rack >> > > > > > >> > > > >> > >>> > > > > > information whenever possible but >> not >> > > make >> > > > > it a >> > > > > > >> hard >> > > > > > >> > > > >> > >>> requirement. >> > > > > > >> > > > >> > >>> > To >> > > > > > >> > > > >> > >>> > > > > extend >> > > > > > >> > > > >> > >>> > > > > > Gwen's example, one badly configured >> > > broker >> > > > > > >> should >> > > > > > >> > not >> > > > > > >> > > > >> > degrade >> > > > > > >> > > > >> > >>> > topic >> > > > > > >> > > > >> > >>> > > > > > creation for the entire cluster. >> > > > > > >> > > > >> > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > > 2. Upgrade scenario - Can you add a >> > > section >> > > > > on >> > > > > > >> the >> > > > > > >> > > > upgrade >> > > > > > >> > > > >> > >>> piece to >> > > > > > >> > > > >> > >>> > > > > confirm >> > > > > > >> > > > >> > >>> > > > > > that old clients will not see >> errors? I >> > > > > believe >> > > > > > >> > > > >> > >>> > > > > ZookeeperConsumerConnector >> > > > > > >> > > > >> > >>> > > > > > reads the Broker objects from ZK. I >> > > wanted >> > > > to >> > > > > > >> > confirm >> > > > > > >> > > > that >> > > > > > >> > > > >> > this >> > > > > > >> > > > >> > >>> > will >> > > > > > >> > > > >> > >>> > > > not >> > > > > > >> > > > >> > >>> > > > > > cause any problems. >> > > > > > >> > > > >> > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > > 3. Could you elaborate your proposed >> > > > changes >> > > > > to >> > > > > > >> the >> > > > > > >> > > > >> > >>> > > > UpdateMetadataRequest >> > > > > > >> > > > >> > >>> > > > > > in the "Public Interfaces" section? >> > > > > > Personally, I >> > > > > > >> > find >> > > > > > >> > > > >> this >> > > > > > >> > > > >> > >>> format >> > > > > > >> > > > >> > >>> > > easy >> > > > > > >> > > > >> > >>> > > > > to >> > > > > > >> > > > >> > >>> > > > > > read in terms of wire protocol >> changes: >> > > > > > >> > > > >> > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > >> > > > > > >> > > > >> > >>> > > > >> > > > > > >> > > > >> > >>> > > >> > > > > > >> > > > >> > >>> > >> > > > > > >> > > > >> > >>> >> > > > > > >> > > > >> > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> > > >> > > > > > >> > >> > > > > > >> >> > > > > > >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-CreateTopicRequest >> > > > > > >> > > > >> > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > > Aditya >> > > > > > >> > > > >> > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > > On Fri, Oct 16, 2015 at 3:45 PM, >> Allen >> > > > Wang < >> > > > > > >> > > > >> > >>> allenxw...@gmail.com> >> > > > > > >> > > > >> > >>> > > > > wrote: >> > > > > > >> > > > >> > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > > > KIP is updated include rack as an >> > > > optional >> > > > > > >> > property >> > > > > > >> > > > for >> > > > > > >> > > > >> > >>> broker. >> > > > > > >> > > > >> > >>> > > > Please >> > > > > > >> > > > >> > >>> > > > > > take >> > > > > > >> > > > >> > >>> > > > > > > a look and let me know if more >> > details >> > > > are >> > > > > > >> needed. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > > >> > > > >> > >>> > > > > > > For the case where some brokers >> have >> > > rack >> > > > > and >> > > > > > >> some >> > > > > > >> > > do >> > > > > > >> > > > >> not, >> > > > > > >> > > > >> > >>> the >> > > > > > >> > > > >> > >>> > > > current >> > > > > > >> > > > >> > >>> > > > > > KIP >> > > > > > >> > > > >> > >>> > > > > > > uses the fail-fast behavior. If >> there >> > > are >> > > > > > >> > concerns, >> > > > > > >> > > we >> > > > > > >> > > > >> can >> > > > > > >> > > > >> > >>> > further >> > > > > > >> > > > >> > >>> > > > > > discuss >> > > > > > >> > > > >> > >>> > > > > > > this in the email thread or next >> > > hangout. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > > >> > > > >> > >>> > > > > > > On Thu, Oct 15, 2015 at 10:42 AM, >> > Allen >> > > > > Wang >> > > > > > < >> > > > > > >> > > > >> > >>> > allenxw...@gmail.com >> > > > > > >> > > > >> > >>> > > > >> > > > > > >> > > > >> > >>> > > > > > wrote: >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > > >> > > > >> > >>> > > > > > > > That's a good question. I can >> think >> > > of >> > > > > > three >> > > > > > >> > > actions >> > > > > > >> > > > >> if >> > > > > > >> > > > >> > the >> > > > > > >> > > > >> > >>> > rack >> > > > > > >> > > > >> > >>> > > > > > > > information is incomplete: >> > > > > > >> > > > >> > >>> > > > > > > > >> > > > > > >> > > > >> > >>> > > > > > > > 1. Treat the node without rack >> as >> > if >> > > it >> > > > > is >> > > > > > on >> > > > > > >> > its >> > > > > > >> > > > >> unique >> > > > > > >> > > > >> > >>> rack >> > > > > > >> > > > >> > >>> > > > > > > > 2. Disregard all rack >> information >> > and >> > > > > > >> fallback >> > > > > > >> > to >> > > > > > >> > > > >> current >> > > > > > >> > > > >> > >>> > > algorithm >> > > > > > >> > > > >> > >>> > > > > > > > 3. Fail-fast >> > > > > > >> > > > >> > >>> > > > > > > > >> > > > > > >> > > > >> > >>> > > > > > > > Now I think about it, one and >> three >> > > > make >> > > > > > more >> > > > > > >> > > sense. >> > > > > > >> > > > >> The >> > > > > > >> > > > >> > >>> reason >> > > > > > >> > > > >> > >>> > > for >> > > > > > >> > > > >> > >>> > > > > > > > fail-fast is that user mistake >> for >> > > not >> > > > > > >> providing >> > > > > > >> > > the >> > > > > > >> > > > >> rack >> > > > > > >> > > > >> > >>> may >> > > > > > >> > > > >> > >>> > > never >> > > > > > >> > > > >> > >>> > > > > be >> > > > > > >> > > > >> > >>> > > > > > > > found if we tolerate that and >> the >> > > > > > assignment >> > > > > > >> may >> > > > > > >> > > not >> > > > > > >> > > > >> be >> > > > > > >> > > > >> > >>> rack >> > > > > > >> > > > >> > >>> > > aware >> > > > > > >> > > > >> > >>> > > > as >> > > > > > >> > > > >> > >>> > > > > > the >> > > > > > >> > > > >> > >>> > > > > > > > user has expected and this >> creates >> > > > debug >> > > > > > >> > problems >> > > > > > >> > > > when >> > > > > > >> > > > >> > >>> things >> > > > > > >> > > > >> > >>> > > fail. >> > > > > > >> > > > >> > >>> > > > > > > > >> > > > > > >> > > > >> > >>> > > > > > > > What do you think? If not >> > fail-fast, >> > > is >> > > > > > there >> > > > > > >> > > anyway >> > > > > > >> > > > >> we >> > > > > > >> > > > >> > can >> > > > > > >> > > > >> > >>> > make >> > > > > > >> > > > >> > >>> > > > the >> > > > > > >> > > > >> > >>> > > > > > user >> > > > > > >> > > > >> > >>> > > > > > > > error standing out? >> > > > > > >> > > > >> > >>> > > > > > > > >> > > > > > >> > > > >> > >>> > > > > > > > >> > > > > > >> > > > >> > >>> > > > > > > > On Thu, Oct 15, 2015 at 10:17 >> AM, >> > > Gwen >> > > > > > >> Shapira < >> > > > > > >> > > > >> > >>> > > g...@confluent.io> >> > > > > > >> > > > >> > >>> > > > > > > wrote: >> > > > > > >> > > > >> > >>> > > > > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> Thanks! Just to clarify, when >> some >> > > > > brokers >> > > > > > >> have >> > > > > > >> > > > rack >> > > > > > >> > > > >> > >>> > assignment >> > > > > > >> > > > >> > >>> > > > and >> > > > > > >> > > > >> > >>> > > > > > some >> > > > > > >> > > > >> > >>> > > > > > > >> don't, do we act like none of >> them >> > > > have >> > > > > > it? >> > > > > > >> or >> > > > > > >> > > like >> > > > > > >> > > > >> > those >> > > > > > >> > > > >> > >>> > > without >> > > > > > >> > > > >> > >>> > > > > > > >> assignment are in their own >> rack? >> > > > > > >> > > > >> > >>> > > > > > > >> >> > > > > > >> > > > >> > >>> > > > > > > >> The first scenario is good when >> > > first >> > > > > > >> setting >> > > > > > >> > up >> > > > > > >> > > > >> > >>> > rack-awareness, >> > > > > > >> > > > >> > >>> > > > but >> > > > > > >> > > > >> > >>> > > > > > the >> > > > > > >> > > > >> > >>> > > > > > > >> second makes more sense for >> > on-going >> > > > > > >> > maintenance >> > > > > > >> > > (I >> > > > > > >> > > > >> can >> > > > > > >> > > > >> > >>> > totally >> > > > > > >> > > > >> > >>> > > > see >> > > > > > >> > > > >> > >>> > > > > > > >> someone >> > > > > > >> > > > >> > >>> > > > > > > >> adding a node and forgetting to >> > set >> > > > the >> > > > > > rack >> > > > > > >> > > > >> property, >> > > > > > >> > > > >> > we >> > > > > > >> > > > >> > >>> > don't >> > > > > > >> > > > >> > >>> > > > want >> > > > > > >> > > > >> > >>> > > > > > > this >> > > > > > >> > > > >> > >>> > > > > > > >> to change behavior for anything >> > > except >> > > > > the >> > > > > > >> new >> > > > > > >> > > > node). >> > > > > > >> > > > >> > >>> > > > > > > >> >> > > > > > >> > > > >> > >>> > > > > > > >> What do you think? >> > > > > > >> > > > >> > >>> > > > > > > >> >> > > > > > >> > > > >> > >>> > > > > > > >> Gwen >> > > > > > >> > > > >> > >>> > > > > > > >> >> > > > > > >> > > > >> > >>> > > > > > > >> On Thu, Oct 15, 2015 at 10:13 >> AM, >> > > > Allen >> > > > > > >> Wang < >> > > > > > >> > > > >> > >>> > > > allenxw...@gmail.com> >> > > > > > >> > > > >> > >>> > > > > > > >> wrote: >> > > > > > >> > > > >> > >>> > > > > > > >> >> > > > > > >> > > > >> > >>> > > > > > > >> > For scenario 1: >> > > > > > >> > > > >> > >>> > > > > > > >> > >> > > > > > >> > > > >> > >>> > > > > > > >> > - Add the rack information to >> > > broker >> > > > > > >> property >> > > > > > >> > > > file >> > > > > > >> > > > >> or >> > > > > > >> > > > >> > >>> > > > dynamically >> > > > > > >> > > > >> > >>> > > > > > set >> > > > > > >> > > > >> > >>> > > > > > > >> it in >> > > > > > >> > > > >> > >>> > > > > > > >> > the wrapper code to bootstrap >> > > Kafka >> > > > > > >> server. >> > > > > > >> > You >> > > > > > >> > > > >> would >> > > > > > >> > > > >> > do >> > > > > > >> > > > >> > >>> > that >> > > > > > >> > > > >> > >>> > > > for >> > > > > > >> > > > >> > >>> > > > > > all >> > > > > > >> > > > >> > >>> > > > > > > >> > brokers and restart the >> brokers >> > > one >> > > > by >> > > > > > >> one. >> > > > > > >> > > > >> > >>> > > > > > > >> > >> > > > > > >> > > > >> > >>> > > > > > > >> > In this scenario, the >> complete >> > > > broker >> > > > > to >> > > > > > >> rack >> > > > > > >> > > > >> mapping >> > > > > > >> > > > >> > >>> may >> > > > > > >> > > > >> > >>> > not >> > > > > > >> > > > >> > >>> > > be >> > > > > > >> > > > >> > >>> > > > > > > >> available >> > > > > > >> > > > >> > >>> > > > > > > >> > until every broker is >> restarted. >> > > > > During >> > > > > > >> that >> > > > > > >> > > time >> > > > > > >> > > > >> we >> > > > > > >> > > > >> > >>> fall >> > > > > > >> > > > >> > >>> > back >> > > > > > >> > > > >> > >>> > > > to >> > > > > > >> > > > >> > >>> > > > > > > >> default >> > > > > > >> > > > >> > >>> > > > > > > >> > replica assignment algorithm. >> > > > > > >> > > > >> > >>> > > > > > > >> > >> > > > > > >> > > > >> > >>> > > > > > > >> > For scenario 2: >> > > > > > >> > > > >> > >>> > > > > > > >> > >> > > > > > >> > > > >> > >>> > > > > > > >> > - Add the rack information to >> > > broker >> > > > > > >> property >> > > > > > >> > > > file >> > > > > > >> > > > >> or >> > > > > > >> > > > >> > >>> > > > dynamically >> > > > > > >> > > > >> > >>> > > > > > set >> > > > > > >> > > > >> > >>> > > > > > > >> it in >> > > > > > >> > > > >> > >>> > > > > > > >> > the wrapper code and start >> the >> > > > broker. >> > > > > > >> > > > >> > >>> > > > > > > >> > >> > > > > > >> > > > >> > >>> > > > > > > >> > >> > > > > > >> > > > >> > >>> > > > > > > >> > On Wed, Oct 14, 2015 at 2:36 >> PM, >> > > > Gwen >> > > > > > >> > Shapira < >> > > > > > >> > > > >> > >>> > > > g...@confluent.io> >> > > > > > >> > > > >> > >>> > > > > > > >> wrote: >> > > > > > >> > > > >> > >>> > > > > > > >> > >> > > > > > >> > > > >> > >>> > > > > > > >> > > Can you clarify the >> workflow >> > for >> > > > the >> > > > > > >> > > following >> > > > > > >> > > > >> > >>> scenarios: >> > > > > > >> > > > >> > >>> > > > > > > >> > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > 1. I currently have 6 >> brokers >> > > and >> > > > > want >> > > > > > >> to >> > > > > > >> > add >> > > > > > >> > > > >> rack >> > > > > > >> > > > >> > >>> > > information >> > > > > > >> > > > >> > >>> > > > > for >> > > > > > >> > > > >> > >>> > > > > > > >> each >> > > > > > >> > > > >> > >>> > > > > > > >> > > 2. I'm adding a new broker >> > and I >> > > > > want >> > > > > > to >> > > > > > >> > > > specify >> > > > > > >> > > > >> > which >> > > > > > >> > > > >> > >>> > rack >> > > > > > >> > > > >> > >>> > > it >> > > > > > >> > > > >> > >>> > > > > > > >> belongs on >> > > > > > >> > > > >> > >>> > > > > > > >> > > while adding it. >> > > > > > >> > > > >> > >>> > > > > > > >> > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > Thanks! >> > > > > > >> > > > >> > >>> > > > > > > >> > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > On Tue, Oct 13, 2015 at >> 2:21 >> > PM, >> > > > > Allen >> > > > > > >> > Wang < >> > > > > > >> > > > >> > >>> > > > > allenxw...@gmail.com >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > wrote: >> > > > > > >> > > > >> > >>> > > > > > > >> > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > We discussed the KIP in >> the >> > > > > hangout >> > > > > > >> > today. >> > > > > > >> > > > The >> > > > > > >> > > > >> > >>> > > > recommendation >> > > > > > >> > > > >> > >>> > > > > is >> > > > > > >> > > > >> > >>> > > > > > > to >> > > > > > >> > > > >> > >>> > > > > > > >> > make >> > > > > > >> > > > >> > >>> > > > > > > >> > > > rack as a broker >> property in >> > > > > > >> ZooKeeper. >> > > > > > >> > For >> > > > > > >> > > > >> users >> > > > > > >> > > > >> > >>> with >> > > > > > >> > > > >> > >>> > > > > existing >> > > > > > >> > > > >> > >>> > > > > > > rack >> > > > > > >> > > > >> > >>> > > > > > > >> > > > information stored >> > somewhere, >> > > > they >> > > > > > >> would >> > > > > > >> > > need >> > > > > > >> > > > >> to >> > > > > > >> > > > >> > >>> > retrieve >> > > > > > >> > > > >> > >>> > > > the >> > > > > > >> > > > >> > >>> > > > > > > >> > information >> > > > > > >> > > > >> > >>> > > > > > > >> > > > at broker start up and >> > > > dynamically >> > > > > > set >> > > > > > >> > the >> > > > > > >> > > > rack >> > > > > > >> > > > >> > >>> > property, >> > > > > > >> > > > >> > >>> > > > > which >> > > > > > >> > > > >> > >>> > > > > > > can >> > > > > > >> > > > >> > >>> > > > > > > >> be >> > > > > > >> > > > >> > >>> > > > > > > >> > > > implemented as a wrapper >> to >> > > > > > bootstrap >> > > > > > >> > > broker. >> > > > > > >> > > > >> > There >> > > > > > >> > > > >> > >>> will >> > > > > > >> > > > >> > >>> > > be >> > > > > > >> > > > >> > >>> > > > no >> > > > > > >> > > > >> > >>> > > > > > > >> > interface >> > > > > > >> > > > >> > >>> > > > > > > >> > > or >> > > > > > >> > > > >> > >>> > > > > > > >> > > > pluggable implementation >> to >> > > > > retrieve >> > > > > > >> the >> > > > > > >> > > rack >> > > > > > >> > > > >> > >>> > information. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > The assumption is that >> you >> > > > always >> > > > > > >> need to >> > > > > > >> > > > >> restart >> > > > > > >> > > > >> > >>> the >> > > > > > >> > > > >> > >>> > > broker >> > > > > > >> > > > >> > >>> > > > > to >> > > > > > >> > > > >> > >>> > > > > > > >> make a >> > > > > > >> > > > >> > >>> > > > > > > >> > > > change to the rack. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > Once the rack becomes a >> > broker >> > > > > > >> property, >> > > > > > >> > it >> > > > > > >> > > > >> will >> > > > > > >> > > > >> > be >> > > > > > >> > > > >> > >>> > > possible >> > > > > > >> > > > >> > >>> > > > > to >> > > > > > >> > > > >> > >>> > > > > > > make >> > > > > > >> > > > >> > >>> > > > > > > >> > rack >> > > > > > >> > > > >> > >>> > > > > > > >> > > > part of the meta data to >> > help >> > > > the >> > > > > > >> > consumer >> > > > > > >> > > > >> choose >> > > > > > >> > > > >> > >>> which >> > > > > > >> > > > >> > >>> > in >> > > > > > >> > > > >> > >>> > > > > sync >> > > > > > >> > > > >> > >>> > > > > > > >> replica >> > > > > > >> > > > >> > >>> > > > > > > >> > > to >> > > > > > >> > > > >> > >>> > > > > > > >> > > > consume from as part of >> the >> > > > future >> > > > > > >> > consumer >> > > > > > >> > > > >> > >>> enhancement. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > I will update the KIP. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > Thanks, >> > > > > > >> > > > >> > >>> > > > > > > >> > > > Allen >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > On Thu, Oct 8, 2015 at >> 9:23 >> > > AM, >> > > > > > Allen >> > > > > > >> > Wang >> > > > > > >> > > < >> > > > > > >> > > > >> > >>> > > > > > allenxw...@gmail.com> >> > > > > > >> > > > >> > >>> > > > > > > >> > wrote: >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > I attended Tuesday's >> KIP >> > > > hangout >> > > > > > but >> > > > > > >> > this >> > > > > > >> > > > KIP >> > > > > > >> > > > >> > was >> > > > > > >> > > > >> > >>> not >> > > > > > >> > > > >> > >>> > > > > > discussed >> > > > > > >> > > > >> > >>> > > > > > > >> due >> > > > > > >> > > > >> > >>> > > > > > > >> > to >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > time constraint. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > However, after hearing >> > > > > discussion >> > > > > > of >> > > > > > >> > > > KIP-35, >> > > > > > >> > > > >> I >> > > > > > >> > > > >> > >>> have >> > > > > > >> > > > >> > >>> > the >> > > > > > >> > > > >> > >>> > > > > > feeling >> > > > > > >> > > > >> > >>> > > > > > > >> that >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > incompatibility >> (caused by >> > > new >> > > > > > >> broker >> > > > > > >> > > > >> property) >> > > > > > >> > > > >> > >>> > between >> > > > > > >> > > > >> > >>> > > > > > brokers >> > > > > > >> > > > >> > >>> > > > > > > >> with >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > different versions >> will >> > be >> > > > > solved >> > > > > > >> > there. >> > > > > > >> > > > In >> > > > > > >> > > > >> > >>> addition, >> > > > > > >> > > > >> > >>> > > > > having >> > > > > > >> > > > >> > >>> > > > > > > >> stack >> > > > > > >> > > > >> > >>> > > > > > > >> > in >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > broker property as meta >> > data >> > > > may >> > > > > > >> also >> > > > > > >> > > help >> > > > > > >> > > > >> > >>> consumers >> > > > > > >> > > > >> > >>> > in >> > > > > > >> > > > >> > >>> > > > the >> > > > > > >> > > > >> > >>> > > > > > > >> future. >> > > > > > >> > > > >> > >>> > > > > > > >> > So >> > > > > > >> > > > >> > >>> > > > > > > >> > > I >> > > > > > >> > > > >> > >>> > > > > > > >> > > > am >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > open to adding stack >> > > property >> > > > to >> > > > > > >> > broker. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > Hopefully we can >> discuss >> > > this >> > > > in >> > > > > > the >> > > > > > >> > next >> > > > > > >> > > > KIP >> > > > > > >> > > > >> > >>> hangout. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > On Wed, Sep 30, 2015 at >> > 2:46 >> > > > PM, >> > > > > > >> Allen >> > > > > > >> > > > Wang < >> > > > > > >> > > > >> > >>> > > > > > > allenxw...@gmail.com >> > > > > > >> > > > >> > >>> > > > > > > >> > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > wrote: >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> Can you send me the >> > > > information >> > > > > > on >> > > > > > >> the >> > > > > > >> > > > next >> > > > > > >> > > > >> KIP >> > > > > > >> > > > >> > >>> > > hangout? >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> Currently the >> broker-rack >> > > > > mapping >> > > > > > >> is >> > > > > > >> > not >> > > > > > >> > > > >> > cached. >> > > > > > >> > > > >> > >>> In >> > > > > > >> > > > >> > >>> > > > > > KafkaApis, >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> >> RackLocator.getRackInfo() >> > > is >> > > > > > called >> > > > > > >> > each >> > > > > > >> > > > >> time >> > > > > > >> > > > >> > the >> > > > > > >> > > > >> > >>> > > mapping >> > > > > > >> > > > >> > >>> > > > > is >> > > > > > >> > > > >> > >>> > > > > > > >> needed >> > > > > > >> > > > >> > >>> > > > > > > >> > > for >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> auto topic creation. >> This >> > > > will >> > > > > > >> ensure >> > > > > > >> > > > latest >> > > > > > >> > > > >> > >>> mapping >> > > > > > >> > > > >> > >>> > is >> > > > > > >> > > > >> > >>> > > > > used >> > > > > > >> > > > >> > >>> > > > > > at >> > > > > > >> > > > >> > >>> > > > > > > >> any >> > > > > > >> > > > >> > >>> > > > > > > >> > > > time. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> The ability to get the >> > > > complete >> > > > > > >> > mapping >> > > > > > >> > > > >> makes >> > > > > > >> > > > >> > it >> > > > > > >> > > > >> > >>> > simple >> > > > > > >> > > > >> > >>> > > > to >> > > > > > >> > > > >> > >>> > > > > > > reuse >> > > > > > >> > > > >> > >>> > > > > > > >> the >> > > > > > >> > > > >> > >>> > > > > > > >> > > > same >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> interface in command >> line >> > > > > tools. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> On Wed, Sep 30, 2015 >> at >> > > 11:01 >> > > > > AM, >> > > > > > >> > Aditya >> > > > > > >> > > > >> > >>> Auradkar < >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> >> > > > aaurad...@linkedin.com.invalid >> > > > > > >> > > > > > >> > wrote: >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> Perhaps we discuss >> this >> > > > during >> > > > > > the >> > > > > > >> > next >> > > > > > >> > > > KIP >> > > > > > >> > > > >> > >>> hangout? >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> I do see that a >> > pluggable >> > > > rack >> > > > > > >> > locator >> > > > > > >> > > > can >> > > > > > >> > > > >> be >> > > > > > >> > > > >> > >>> useful >> > > > > > >> > > > >> > >>> > > > but I >> > > > > > >> > > > >> > >>> > > > > > do >> > > > > > >> > > > >> > >>> > > > > > > >> see a >> > > > > > >> > > > >> > >>> > > > > > > >> > > few >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> concerns: >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> - The RackLocator (as >> > > > > described >> > > > > > in >> > > > > > >> > the >> > > > > > >> > > > >> > >>> document), >> > > > > > >> > > > >> > >>> > > > implies >> > > > > > >> > > > >> > >>> > > > > > that >> > > > > > >> > > > >> > >>> > > > > > > >> it >> > > > > > >> > > > >> > >>> > > > > > > >> > can >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> discover rack >> > information >> > > > for >> > > > > > any >> > > > > > >> > node >> > > > > > >> > > in >> > > > > > >> > > > >> the >> > > > > > >> > > > >> > >>> > cluster. >> > > > > > >> > > > >> > >>> > > > How >> > > > > > >> > > > >> > >>> > > > > > > does >> > > > > > >> > > > >> > >>> > > > > > > >> it >> > > > > > >> > > > >> > >>> > > > > > > >> > > deal >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> with rack location >> > > changes? >> > > > > For >> > > > > > >> > > example, >> > > > > > >> > > > >> if I >> > > > > > >> > > > >> > >>> moved >> > > > > > >> > > > >> > >>> > > > broker >> > > > > > >> > > > >> > >>> > > > > > id >> > > > > > >> > > > >> > >>> > > > > > > >> (1) >> > > > > > >> > > > >> > >>> > > > > > > >> > > from >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> rack >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> X to Y, I only have >> to >> > > start >> > > > > > that >> > > > > > >> > > broker >> > > > > > >> > > > >> with >> > > > > > >> > > > >> > a >> > > > > > >> > > > >> > >>> > newer >> > > > > > >> > > > >> > >>> > > > rack >> > > > > > >> > > > >> > >>> > > > > > > >> config. >> > > > > > >> > > > >> > >>> > > > > > > >> > If >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> RackLocator discovers >> > > broker >> > > > > -> >> > > > > > >> rack >> > > > > > >> > > > >> > >>> information at >> > > > > > >> > > > >> > >>> > > > start >> > > > > > >> > > > >> > >>> > > > > up >> > > > > > >> > > > >> > >>> > > > > > > >> time, >> > > > > > >> > > > >> > >>> > > > > > > >> > > any >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> change to a broker >> will >> > > > > require >> > > > > > >> > > bouncing >> > > > > > >> > > > >> the >> > > > > > >> > > > >> > >>> entire >> > > > > > >> > > > >> > >>> > > > > cluster >> > > > > > >> > > > >> > >>> > > > > > > >> since >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> createTopic requests >> can >> > > be >> > > > > sent >> > > > > > >> to >> > > > > > >> > any >> > > > > > >> > > > >> node >> > > > > > >> > > > >> > in >> > > > > > >> > > > >> > >>> the >> > > > > > >> > > > >> > >>> > > > > cluster. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> For this reason it >> may >> > be >> > > > > > simpler >> > > > > > >> to >> > > > > > >> > > have >> > > > > > >> > > > >> each >> > > > > > >> > > > >> > >>> node >> > > > > > >> > > > >> > >>> > be >> > > > > > >> > > > >> > >>> > > > > aware >> > > > > > >> > > > >> > >>> > > > > > > of >> > > > > > >> > > > >> > >>> > > > > > > >> its >> > > > > > >> > > > >> > >>> > > > > > > >> > > own >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> rack and persist it >> in >> > ZK >> > > > > during >> > > > > > >> > start >> > > > > > >> > > up >> > > > > > >> > > > >> > time. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> - A pluggable >> > RackLocator >> > > > > relies >> > > > > > >> on >> > > > > > >> > an >> > > > > > >> > > > >> > external >> > > > > > >> > > > >> > >>> > > service >> > > > > > >> > > > >> > >>> > > > > > being >> > > > > > >> > > > >> > >>> > > > > > > >> > > available >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> to >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> serve rack >> information. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> Out of curiosity, I >> > looked >> > > > up >> > > > > > how >> > > > > > >> a >> > > > > > >> > > > couple >> > > > > > >> > > > >> of >> > > > > > >> > > > >> > >>> other >> > > > > > >> > > > >> > >>> > > > > systems >> > > > > > >> > > > >> > >>> > > > > > > deal >> > > > > > >> > > > >> > >>> > > > > > > >> > with >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> zone/rack awareness. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> For Cassandra some >> > > > interesting >> > > > > > >> modes >> > > > > > >> > > are: >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> (Property File >> > > > configuration) >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > >> > > > > > >> > > > >> > >>> > > > > > > >> > >> > > > > > >> > > > >> > >>> > > > > > > >> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > > >> > > > >> > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > >> > > > > > >> > > > >> > >>> > > > >> > > > > > >> > > > >> > >>> > > >> > > > > > >> > > > >> > >>> > >> > > > > > >> > > > >> > >>> >> > > > > > >> > > > >> > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> > > >> > > > > > >> > >> > > > > > >> >> > > > > > >> > > > > >> > > > >> > > >> > >> http://docs.datastax.com/en/cassandra/2.0/cassandra/architecture/architectureSnitchPFSnitch_t.html >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> (Dynamic inference) >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > >> > > > > > >> > > > >> > >>> > > > > > > >> > >> > > > > > >> > > > >> > >>> > > > > > > >> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > > >> > > > >> > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > >> > > > > > >> > > > >> > >>> > > > >> > > > > > >> > > > >> > >>> > > >> > > > > > >> > > > >> > >>> > >> > > > > > >> > > > >> > >>> >> > > > > > >> > > > >> > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> > > >> > > > > > >> > >> > > > > > >> >> > > > > > >> > > > > >> > > > >> > > >> > >> http://docs.datastax.com/en/cassandra/2.0/cassandra/architecture/architectureSnitchRackInf_c.html >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> Voldemort does a >> static >> > > node >> > > > > -> >> > > > > > >> zone >> > > > > > >> > > > >> > assignment >> > > > > > >> > > > >> > >>> > based >> > > > > > >> > > > >> > >>> > > on >> > > > > > >> > > > >> > >>> > > > > > > >> > > configuration. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> Aditya >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> On Wed, Sep 30, 2015 >> at >> > > > 10:05 >> > > > > > AM, >> > > > > > >> > Allen >> > > > > > >> > > > >> Wang < >> > > > > > >> > > > >> > >>> > > > > > > >> allenxw...@gmail.com >> > > > > > >> > > > >> > >>> > > > > > > >> > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> wrote: >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > I would like to >> see if >> > > we >> > > > > can >> > > > > > do >> > > > > > >> > > both: >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > - Make RackLocator >> > > > pluggable >> > > > > > to >> > > > > > >> > > > >> facilitate >> > > > > > >> > > > >> > >>> > migration >> > > > > > >> > > > >> > >>> > > > > with >> > > > > > >> > > > >> > >>> > > > > > > >> > existing >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > broker-rack mapping >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > - Make rack an >> > optional >> > > > > > property >> > > > > > >> > for >> > > > > > >> > > > >> broker. >> > > > > > >> > > > >> > >>> If >> > > > > > >> > > > >> > >>> > rack >> > > > > > >> > > > >> > >>> > > > is >> > > > > > >> > > > >> > >>> > > > > > > >> available >> > > > > > >> > > > >> > >>> > > > > > > >> > > > from >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > broker, treat it as >> > > source >> > > > > of >> > > > > > >> > truth. >> > > > > > >> > > > For >> > > > > > >> > > > >> > users >> > > > > > >> > > > >> > >>> > with >> > > > > > >> > > > >> > >>> > > > > > existing >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> broker-rack >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > mapping somewhere >> > else, >> > > > they >> > > > > > can >> > > > > > >> > use >> > > > > > >> > > > the >> > > > > > >> > > > >> > >>> pluggable >> > > > > > >> > > > >> > >>> > > way >> > > > > > >> > > > >> > >>> > > > > or >> > > > > > >> > > > >> > >>> > > > > > > they >> > > > > > >> > > > >> > >>> > > > > > > >> > can >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> transfer >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > the mapping to the >> > > broker >> > > > > rack >> > > > > > >> > > > property. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > One thing I am not >> > sure >> > > is >> > > > > > what >> > > > > > >> > > happens >> > > > > > >> > > > >> at >> > > > > > >> > > > >> > >>> rolling >> > > > > > >> > > > >> > >>> > > > > upgrade >> > > > > > >> > > > >> > >>> > > > > > > >> when >> > > > > > >> > > > >> > >>> > > > > > > >> > we >> > > > > > >> > > > >> > >>> > > > > > > >> > > > have >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > rack as a broker >> > > property. >> > > > > For >> > > > > > >> > > brokers >> > > > > > >> > > > >> with >> > > > > > >> > > > >> > >>> older >> > > > > > >> > > > >> > >>> > > > > version >> > > > > > >> > > > >> > >>> > > > > > of >> > > > > > >> > > > >> > >>> > > > > > > >> > Kafka, >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> will it >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > cause problem for >> > them? >> > > If >> > > > > so, >> > > > > > >> is >> > > > > > >> > > there >> > > > > > >> > > > >> any >> > > > > > >> > > > >> > >>> > > > workaround? >> > > > > > >> > > > >> > >>> > > > > I >> > > > > > >> > > > >> > >>> > > > > > > also >> > > > > > >> > > > >> > >>> > > > > > > >> > > think >> > > > > > >> > > > >> > >>> > > > > > > >> > > > it >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > would be better >> not to >> > > > have >> > > > > > >> rack in >> > > > > > >> > > the >> > > > > > >> > > > >> > >>> controller >> > > > > > >> > > > >> > >>> > > > wire >> > > > > > >> > > > >> > >>> > > > > > > >> protocol >> > > > > > >> > > > >> > >>> > > > > > > >> > > but >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> not >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > sure if it is >> > > achievable. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > Thanks, >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > Allen >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > On Mon, Sep 28, >> 2015 >> > at >> > > > 4:55 >> > > > > > PM, >> > > > > > >> > Todd >> > > > > > >> > > > >> > Palino < >> > > > > > >> > > > >> > >>> > > > > > > >> tpal...@gmail.com> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> wrote: >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > I tend to like >> the >> > > idea >> > > > > of a >> > > > > > >> > > > pluggable >> > > > > > >> > > > >> > >>> locator. >> > > > > > >> > > > >> > >>> > > For >> > > > > > >> > > > >> > >>> > > > > > > >> example, we >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> already >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > have an interface >> > for >> > > > > > >> discovering >> > > > > > >> > > > >> > >>> information >> > > > > > >> > > > >> > >>> > > about >> > > > > > >> > > > >> > >>> > > > > the >> > > > > > >> > > > >> > >>> > > > > > > >> > physical >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> location >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > of servers. I >> don't >> > > > relish >> > > > > > the >> > > > > > >> > idea >> > > > > > >> > > > of >> > > > > > >> > > > >> > >>> having to >> > > > > > >> > > > >> > >>> > > > > > maintain >> > > > > > >> > > > >> > >>> > > > > > > >> data >> > > > > > >> > > > >> > >>> > > > > > > >> > in >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > multiple >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > places. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > -Todd >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > On Mon, Sep 28, >> 2015 >> > > at >> > > > > 4:48 >> > > > > > >> PM, >> > > > > > >> > > > Aditya >> > > > > > >> > > > >> > >>> > Auradkar < >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > >> > > > > > aaurad...@linkedin.com.invalid >> > > > > > >> > >> > > > > > >> > > > wrote: >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > Thanks for >> > starting >> > > > this >> > > > > > KIP >> > > > > > >> > > Allen. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > I agree with >> Gwen >> > > that >> > > > > > >> having a >> > > > > > >> > > > >> > >>> RackLocator >> > > > > > >> > > > >> > >>> > > class >> > > > > > >> > > > >> > >>> > > > > that >> > > > > > >> > > > >> > >>> > > > > > > is >> > > > > > >> > > > >> > >>> > > > > > > >> > > > pluggable >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > seems >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > to be too >> complex. >> > > The >> > > > > KIP >> > > > > > >> > refers >> > > > > > >> > > > to >> > > > > > >> > > > >> > >>> > potentially >> > > > > > >> > > > >> > >>> > > > > > non-ZK >> > > > > > >> > > > >> > >>> > > > > > > >> > storage >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> for the >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > rack info >> which I >> > > > don't >> > > > > > >> think >> > > > > > >> > is >> > > > > > >> > > > >> > >>> necessary. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > Perhaps we can >> > > persist >> > > > > > this >> > > > > > >> > info >> > > > > > >> > > in >> > > > > > >> > > > >> zk >> > > > > > >> > > > >> > >>> under >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> >> /brokers/ids/<broker_id> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > similar to >> other >> > > > broker >> > > > > > >> > > properties >> > > > > > >> > > > >> and >> > > > > > >> > > > >> > >>> add a >> > > > > > >> > > > >> > >>> > > > config >> > > > > > >> > > > >> > >>> > > > > in >> > > > > > >> > > > >> > >>> > > > > > > >> > > > KafkaConfig >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > called >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > "rack". >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > > >> > > > >> >> > {"jmx_port":-1,"endpoints":[...],"host":"xxx","port":yyy, >> > > > > > >> > > > >> > >>> > > > > > > >> > > "rack": >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > "abc"} >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > Aditya >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > On Mon, Sep 28, >> > 2015 >> > > > at >> > > > > > 2:30 >> > > > > > >> > PM, >> > > > > > >> > > > Gwen >> > > > > > >> > > > >> > >>> Shapira >> > > > > > >> > > > >> > >>> > < >> > > > > > >> > > > >> > >>> > > > > > > >> > > g...@confluent.io >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > wrote: >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > Hi, >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > First, thanks >> > for >> > > > > > putting >> > > > > > >> > out a >> > > > > > >> > > > KIP >> > > > > > >> > > > >> > for >> > > > > > >> > > > >> > >>> > this. >> > > > > > >> > > > >> > >>> > > > This >> > > > > > >> > > > >> > >>> > > > > > is >> > > > > > >> > > > >> > >>> > > > > > > >> super >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> important >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > for >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > production >> > > > deployments >> > > > > > of >> > > > > > >> > > Kafka. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > Few >> questions: >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > 1) Are we >> sure >> > we >> > > > want >> > > > > > "as >> > > > > > >> > many >> > > > > > >> > > > >> racks >> > > > > > >> > > > >> > as >> > > > > > >> > > > >> > >>> > > > > possible"? >> > > > > > >> > > > >> > >>> > > > > > > I'd >> > > > > > >> > > > >> > >>> > > > > > > >> > want >> > > > > > >> > > > >> > >>> > > > > > > >> > > to >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > balance >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > between >> safety >> > > (more >> > > > > > >> racks) >> > > > > > >> > and >> > > > > > >> > > > >> > network >> > > > > > >> > > > >> > >>> > > > > utilization >> > > > > > >> > > > >> > >>> > > > > > > >> > (traffic >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> within a >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > rack >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > uses the >> > > > > high-bandwidth >> > > > > > >> TOR >> > > > > > >> > > > >> switch). >> > > > > > >> > > > >> > One >> > > > > > >> > > > >> > >>> > > replica >> > > > > > >> > > > >> > >>> > > > > on >> > > > > > >> > > > >> > >>> > > > > > a >> > > > > > >> > > > >> > >>> > > > > > > >> > > different >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> rack >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > and >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > the rest on >> same >> > > > rack >> > > > > > (if >> > > > > > >> > > > possible) >> > > > > > >> > > > >> > >>> sounds >> > > > > > >> > > > >> > >>> > > > better >> > > > > > >> > > > >> > >>> > > > > to >> > > > > > >> > > > >> > >>> > > > > > > me. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > 2) >> Rack-locator >> > > > class >> > > > > > >> seems >> > > > > > >> > > > overly >> > > > > > >> > > > >> > >>> complex >> > > > > > >> > > > >> > >>> > > > > compared >> > > > > > >> > > > >> > >>> > > > > > to >> > > > > > >> > > > >> > >>> > > > > > > >> > > adding a >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > rack.number >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > property to >> the >> > > > broker >> > > > > > >> > > properties >> > > > > > >> > > > >> > file. >> > > > > > >> > > > >> > >>> Why >> > > > > > >> > > > >> > >>> > do >> > > > > > >> > > > >> > >>> > > > we >> > > > > > >> > > > >> > >>> > > > > > want >> > > > > > >> > > > >> > >>> > > > > > > >> > that? >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > Gwen >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > On Mon, Sep >> 28, >> > > 2015 >> > > > > at >> > > > > > >> 12:15 >> > > > > > >> > > PM, >> > > > > > >> > > > >> > Allen >> > > > > > >> > > > >> > >>> > Wang < >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> allenxw...@gmail.com >> > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > wrote: >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > > Hello Kafka >> > > > > > Developers, >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > > I just >> created >> > > > > KIP-36 >> > > > > > >> for >> > > > > > >> > > rack >> > > > > > >> > > > >> aware >> > > > > > >> > > > >> > >>> > replica >> > > > > > >> > > > >> > >>> > > > > > > >> assignment. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > >> > > > > > >> > > > >> > >>> > > > > > > >> > >> > > > > > >> > > > >> > >>> > > > > > > >> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > > >> > > > >> > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > >> > > > > > >> > > > >> > >>> > > > >> > > > > > >> > > > >> > >>> > > >> > > > > > >> > > > >> > >>> > >> > > > > > >> > > > >> > >>> >> > > > > > >> > > > >> > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> > > >> > > > > > >> > >> > > > > > >> >> > > > > > >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-36+Rack+aware+replica+assignment >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > > The goal >> is to >> > > > > utilize >> > > > > > >> the >> > > > > > >> > > > >> isolation >> > > > > > >> > > > >> > >>> > > provided >> > > > > > >> > > > >> > >>> > > > by >> > > > > > >> > > > >> > >>> > > > > > the >> > > > > > >> > > > >> > >>> > > > > > > >> > racks >> > > > > > >> > > > >> > >>> > > > > > > >> > > in >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> data >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > center >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > > and >> distribute >> > > > > > replicas >> > > > > > >> to >> > > > > > >> > > > racks >> > > > > > >> > > > >> to >> > > > > > >> > > > >> > >>> > provide >> > > > > > >> > > > >> > >>> > > > > fault >> > > > > > >> > > > >> > >>> > > > > > > >> > > tolerance. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > > Comments >> are >> > > > > welcome. >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > > Thanks, >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > > Allen >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >>> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > >> > > > > > >> > > > >> > >>> > > > > > > >> > >> > > > > > >> > > > >> > >>> > > > > > > >> >> > > > > > >> > > > >> > >>> > > > > > > > >> > > > > > >> > > > >> > >>> > > > > > > > >> > > > > > >> > > > >> > >>> > > > > > > >> > > > > > >> > > > >> > >>> > > > > > >> > > > > > >> > > > >> > >>> > > > > >> > > > > > >> > > > >> > >>> > > > >> > > > > > >> > > > >> > >>> > > >> > > > > > >> > > > >> > >>> > >> > > > > > >> > > > >> > >>> >> > > > > > >> > > > >> > >>> >> > > > > > >> > > > >> > >>> >> > > > > > >> > > > >> > >>> -- >> > > > > > >> > > > >> > >>> Thanks, >> > > > > > >> > > > >> > >>> Neha >> > > > > > >> > > > >> > >>> >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> > > >> > > > > > >> > > > >> > >> > > > > > >> > > > >> >> > > > > > >> > > > > >> > > > > > >> > > > > >> > > > > > >> > > > >> > > > > > >> > > >> > > > > > >> > >> > > > > > >> >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> > >
