Any thought on my last email about discarding this KIP?
-Matthias On 9/14/18 11:44 AM, Matthias J. Sax wrote: > Hi, > > we recently had a discussion on a different ticket to reduce the size of > the metadata we need to send: > https://issues.apache.org/jira/browse/KAFKA-7149 > > It seems, that we actually don't need to include the number of stores in > the metadata, but that we can compute the number of stores locally on > each instance. > > With this insight, we should still try to exploit this knowledge during > task assignment, however, this would be an internal change that does not > require a KIP. Thus, I think that we can discard this KIP. > > Thoughts? > > > -Matthias > > On 6/10/18 5:20 PM, Matthias J. Sax wrote: >> Richard, >> >> KIP-268 got merged and thus this KIP is unblocked. >> >> I just re-read it and think it needs some updates with regard to the >> upgrade path (ie, you should mention why upgrading is covered). >> >> It would also be useful to discuss how the store information is used >> during assignment. Atm, the KIP only discussed that the information >> should be added, but this is only half of the story from my point of view. >> >> >> -Matthias >> >> On 3/22/18 9:15 PM, Matthias J. Sax wrote: >>> Hi Richard, >>> >>> with KIP-268 in place (should be accepted soon) the upgrade path is >>> covered. Thus, you can update your KIP accordingly, referring to KIP-268. >>> >>> Can you also update your KIP similar to KIP-268 to cover the old and new >>> metadata format? >>> >>> Thanks! >>> >>> -Matthias >>> >>> >>> On 2/24/18 4:07 PM, Richard Yu wrote: >>>> I didn't really get what "upgrade strategy" was at the time that Guozhang >>>> mentioned it, so I wrote the above dialogue from my first understanding. I >>>> changed it to "upgrade strategy must be provided". Currently, however, I do >>>> not have anything in mind to facilitate upgrading older Kafka brokers. If >>>> you have anything in mind, please let me know. >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Sat, Feb 24, 2018 at 3:58 PM, Matthias J. Sax <matth...@confluent.io> >>>> wrote: >>>> >>>>> Thanks a lot for this KIP. >>>>> >>>>> I am not sure what you mean by >>>>> >>>>>> which could potentially break older versions of Kafka brokers >>>>> >>>>> The metadata that is exchange, is not interpreted by the brokers. The >>>>> problem with upgrading the metadata format affect only Kafka Streams >>>>> instances. >>>>> >>>>> If we don't provide an upgrade strategy, changing the metadata format >>>>> required to stop all running application instances, before the instances >>>>> can be restarted with the new code. However, this implies downtime for >>>>> an application and is thus not acceptable. >>>>> >>>>> >>>>> -Matthias >>>>> >>>>> >>>>> On 2/24/18 11:11 AM, Richard Yu wrote: >>>>>> Hi all, >>>>>> >>>>>> I would like to discuss a KIP I've submitted : >>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP- >>>>> 262%3A+Metadata+should+include+number+of+state+stores+for+task >>>>>> >>>>>> >>>>>> Regards, >>>>>> Richard Yu >>>>>> >>>>> >>>>> >>>> >>> >> >
signature.asc
Description: OpenPGP digital signature
