Hi, we recently had a discussion on a different ticket to reduce the size of the metadata we need to send: https://issues.apache.org/jira/browse/KAFKA-7149
It seems, that we actually don't need to include the number of stores in the metadata, but that we can compute the number of stores locally on each instance. With this insight, we should still try to exploit this knowledge during task assignment, however, this would be an internal change that does not require a KIP. Thus, I think that we can discard this KIP. Thoughts? -Matthias On 6/10/18 5:20 PM, Matthias J. Sax wrote: > Richard, > > KIP-268 got merged and thus this KIP is unblocked. > > I just re-read it and think it needs some updates with regard to the > upgrade path (ie, you should mention why upgrading is covered). > > It would also be useful to discuss how the store information is used > during assignment. Atm, the KIP only discussed that the information > should be added, but this is only half of the story from my point of view. > > > -Matthias > > On 3/22/18 9:15 PM, Matthias J. Sax wrote: >> Hi Richard, >> >> with KIP-268 in place (should be accepted soon) the upgrade path is >> covered. Thus, you can update your KIP accordingly, referring to KIP-268. >> >> Can you also update your KIP similar to KIP-268 to cover the old and new >> metadata format? >> >> Thanks! >> >> -Matthias >> >> >> On 2/24/18 4:07 PM, Richard Yu wrote: >>> I didn't really get what "upgrade strategy" was at the time that Guozhang >>> mentioned it, so I wrote the above dialogue from my first understanding. I >>> changed it to "upgrade strategy must be provided". Currently, however, I do >>> not have anything in mind to facilitate upgrading older Kafka brokers. If >>> you have anything in mind, please let me know. >>> >>> >>> >>> >>> >>> >>> On Sat, Feb 24, 2018 at 3:58 PM, Matthias J. Sax <matth...@confluent.io> >>> wrote: >>> >>>> Thanks a lot for this KIP. >>>> >>>> I am not sure what you mean by >>>> >>>>> which could potentially break older versions of Kafka brokers >>>> >>>> The metadata that is exchange, is not interpreted by the brokers. The >>>> problem with upgrading the metadata format affect only Kafka Streams >>>> instances. >>>> >>>> If we don't provide an upgrade strategy, changing the metadata format >>>> required to stop all running application instances, before the instances >>>> can be restarted with the new code. However, this implies downtime for >>>> an application and is thus not acceptable. >>>> >>>> >>>> -Matthias >>>> >>>> >>>> On 2/24/18 11:11 AM, Richard Yu wrote: >>>>> Hi all, >>>>> >>>>> I would like to discuss a KIP I've submitted : >>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP- >>>> 262%3A+Metadata+should+include+number+of+state+stores+for+task >>>>> >>>>> >>>>> Regards, >>>>> Richard Yu >>>>> >>>> >>>> >>> >> >
signature.asc
Description: OpenPGP digital signature
