Hi, Andrew, Thanks for the clarification. Sounds good.
Jun On Thu, Aug 15, 2024 at 2:53 AM Andrew Schofield <andrew_schofi...@live.com> wrote: > Hi Jun, > Thanks for the questions. > > 1. The PartitionMetadata is only stored if there is rack ID information. > So, there are situations in which it’s redundant, but almost always the > PartitionMetadata will be empty. > > 2. They are for subtly different purposes. ShareGroupPartitionMetadata > is essentially a persistent cache of the metadata for topic-partitions > which > can be assigned to group members. ShareGroupStatePartitionMetadata > is keeping track of which partitions have share-group state in the share > coordinator. I prefer having them separate, in particular because it means > the ShareGroupPartitionMetadata matches nicely with consumer groups, > and also because share groups are able to work without a share coordinator > (the persister is pluggable). > > I propose that as we develop the code in the group coordinator which uses > the various ShareGroupState RPCs to talk to the share coordinator, > we will confirm whether the record schemas are optimal > and make a further refinement if required. Let me know if that’s OK. > > Thanks, > Andrew > > > > On 14 Aug 2024, at 00:00, Jun Rao <j...@confluent.io.INVALID> wrote: > > > > Hi, Andrew, > > > > Thanks for the update. A couple of followup questions. > > > > 1. ShareGroupPartitionMetadataValue: Is NumPartitions redundant given > > PartitionMetadata? > > > > 2. I assume that ShareGroupPartitionMetadataValue contains initialized > > partitions. Does that > > make ShareGroupStatePartitionMetadataValue.InitializedTopics redundant? > > > > Jun > > > > On Tue, Aug 13, 2024 at 11:56 AM Andrew Schofield < > andrew_schofi...@live.com> > > wrote: > > > >> Hi, > >> During the implementation of the group management code for KIP-932, > >> we learnt that the decision not to persist assignments for share groups > was > >> making it difficult to fit with the coordinator framework, in particular > >> the > >> use of timeline structures. As a result, I decided to change the KIP to > >> follow the consumer group record scheme much more closely. > >> > >> Now, share groups do persist assignments and member epochs which > >> resolves the problems we were experiencing managing soft state in the > >> coordinator and making group membership reliable across coordinator > >> restarts. > >> > >> I have updated the KIP accordingly and the revised code has been merged. > >> If anyone has any comments, please let me know. > >> > >> Thanks, > >> Andrew > >> > >> > >>> On 21 May 2024, at 15:35, Andrew Schofield <andrew_schofi...@live.com> > >> wrote: > >>> > >>> Hi Jun, > >>> All the client metrics are standard. None are required. > >>> > >>> I’ve updated the KIP accordingly. > >>> > >>> Thanks, > >>> Andrew > >>> > >>>> On 15 May 2024, at 21:36, Jun Rao <j...@confluent.io.INVALID> wrote: > >>>> > >>>> Hi, Andrew, > >>>> > >>>> Thanks for the update. Should we mark whether those metrics are > >>>> standard/required for KIP-714? > >>>> > >>>> Jun > >>>> > >>>> On Tue, May 14, 2024 at 7:31 AM Andrew Schofield < > >> andrew_schofi...@live.com> > >>>> wrote: > >>>> > >>>>> Hi, > >>>>> I have made a small update to the KIP as a result of testing the new > >>>>> share consumer with client telemetry (KIP-714). > >>>>> > >>>>> I’ve added telemetry metric names to the table of client metrics and > >>>>> also updated the metric group names so that the resulting client > >> metrics > >>>>> sent to the broker have consistent names. > >>>>> > >>>>> Thanks, > >>>>> Andrew > >>>>> > >>>>>> On 8 May 2024, at 12:51, Manikumar <manikumar.re...@gmail.com> > wrote: > >>>>>> > >>>>>> Hi Andrew, > >>>>>> > >>>>>> Thanks for the KIP. Great write-up! > >>>>>> > >>>>>> +1 (binding) > >>>>>> > >>>>>> Thanks, > >>>>>> > >>>>>> On Wed, May 8, 2024 at 12:17 PM Satish Duggana < > >> satish.dugg...@gmail.com> > >>>>> wrote: > >>>>>>> > >>>>>>> Hi Andrew, > >>>>>>> Thanks for the nice KIP, it will allow other messaging use cases to > >> be > >>>>>>> onboarded to Kafka. > >>>>>>> > >>>>>>> +1 from me. > >>>>>>> > >>>>>>> Satish. > >>>>>>> > >>>>>>> On Tue, 7 May 2024 at 03:41, Jun Rao <j...@confluent.io.invalid> > >> wrote: > >>>>>>>> > >>>>>>>> Hi, Andrew, > >>>>>>>> > >>>>>>>> Thanks for the KIP. +1 > >>>>>>>> > >>>>>>>> Jun > >>>>>>>> > >>>>>>>> On Mon, Mar 18, 2024 at 11:00 AM Edoardo Comar < > >> edoardli...@gmail.com> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> Thanks Andrew, > >>>>>>>>> > >>>>>>>>> +1 (binding) > >>>>>>>>> > >>>>>>>>> Edo > >>>>>>>>> > >>>>>>>>> On Mon, 18 Mar 2024 at 16:32, Kenneth Eversole > >>>>>>>>> <kevers...@cloudflare.com.invalid> wrote: > >>>>>>>>>> > >>>>>>>>>> Hi Andrew > >>>>>>>>>> > >>>>>>>>>> + 1 (Non-Binding) > >>>>>>>>>> > >>>>>>>>>> This will be great addition to Kafka > >>>>>>>>>> > >>>>>>>>>> On Mon, Mar 18, 2024 at 8:27 AM Apoorv Mittal < > >>>>> apoorvmitta...@gmail.com> > >>>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> Hi Andrew, > >>>>>>>>>>> Thanks for writing the KIP. This is indeed going to be a > valuable > >>>>>>>>> addition > >>>>>>>>>>> to the Kafka, excited to see the KIP. > >>>>>>>>>>> > >>>>>>>>>>> + 1 (Non-Binding) > >>>>>>>>>>> > >>>>>>>>>>> Regards, > >>>>>>>>>>> Apoorv Mittal > >>>>>>>>>>> +44 7721681581 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> On Sun, Mar 17, 2024 at 11:16 PM Andrew Schofield < > >>>>>>>>>>> andrew_schofield_j...@outlook.com> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Hi, > >>>>>>>>>>>> I’ve been working to complete KIP-932 over the past few months > >> and > >>>>>>>>>>>> discussions have quietened down. > >>>>>>>>>>>> > >>>>>>>>>>>> I’d like to open the voting for KIP-932: > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>> > >>>>> > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-932%3A+Queues+for+Kafka > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks, > >>>>>>>>>>>> Andrew > > >
