Hi Jun, Thanks for the questions. 1. The PartitionMetadata is only stored if there is rack ID information. So, there are situations in which it’s redundant, but almost always the PartitionMetadata will be empty.
2. They are for subtly different purposes. ShareGroupPartitionMetadata is essentially a persistent cache of the metadata for topic-partitions which can be assigned to group members. ShareGroupStatePartitionMetadata is keeping track of which partitions have share-group state in the share coordinator. I prefer having them separate, in particular because it means the ShareGroupPartitionMetadata matches nicely with consumer groups, and also because share groups are able to work without a share coordinator (the persister is pluggable). I propose that as we develop the code in the group coordinator which uses the various ShareGroupState RPCs to talk to the share coordinator, we will confirm whether the record schemas are optimal and make a further refinement if required. Let me know if that’s OK. Thanks, Andrew > On 14 Aug 2024, at 00:00, Jun Rao <j...@confluent.io.INVALID> wrote: > > Hi, Andrew, > > Thanks for the update. A couple of followup questions. > > 1. ShareGroupPartitionMetadataValue: Is NumPartitions redundant given > PartitionMetadata? > > 2. I assume that ShareGroupPartitionMetadataValue contains initialized > partitions. Does that > make ShareGroupStatePartitionMetadataValue.InitializedTopics redundant? > > Jun > > On Tue, Aug 13, 2024 at 11:56 AM Andrew Schofield <andrew_schofi...@live.com> > wrote: > >> Hi, >> During the implementation of the group management code for KIP-932, >> we learnt that the decision not to persist assignments for share groups was >> making it difficult to fit with the coordinator framework, in particular >> the >> use of timeline structures. As a result, I decided to change the KIP to >> follow the consumer group record scheme much more closely. >> >> Now, share groups do persist assignments and member epochs which >> resolves the problems we were experiencing managing soft state in the >> coordinator and making group membership reliable across coordinator >> restarts. >> >> I have updated the KIP accordingly and the revised code has been merged. >> If anyone has any comments, please let me know. >> >> Thanks, >> Andrew >> >> >>> On 21 May 2024, at 15:35, Andrew Schofield <andrew_schofi...@live.com> >> wrote: >>> >>> Hi Jun, >>> All the client metrics are standard. None are required. >>> >>> I’ve updated the KIP accordingly. >>> >>> Thanks, >>> Andrew >>> >>>> On 15 May 2024, at 21:36, Jun Rao <j...@confluent.io.INVALID> wrote: >>>> >>>> Hi, Andrew, >>>> >>>> Thanks for the update. Should we mark whether those metrics are >>>> standard/required for KIP-714? >>>> >>>> Jun >>>> >>>> On Tue, May 14, 2024 at 7:31 AM Andrew Schofield < >> andrew_schofi...@live.com> >>>> wrote: >>>> >>>>> Hi, >>>>> I have made a small update to the KIP as a result of testing the new >>>>> share consumer with client telemetry (KIP-714). >>>>> >>>>> I’ve added telemetry metric names to the table of client metrics and >>>>> also updated the metric group names so that the resulting client >> metrics >>>>> sent to the broker have consistent names. >>>>> >>>>> Thanks, >>>>> Andrew >>>>> >>>>>> On 8 May 2024, at 12:51, Manikumar <manikumar.re...@gmail.com> wrote: >>>>>> >>>>>> Hi Andrew, >>>>>> >>>>>> Thanks for the KIP. Great write-up! >>>>>> >>>>>> +1 (binding) >>>>>> >>>>>> Thanks, >>>>>> >>>>>> On Wed, May 8, 2024 at 12:17 PM Satish Duggana < >> satish.dugg...@gmail.com> >>>>> wrote: >>>>>>> >>>>>>> Hi Andrew, >>>>>>> Thanks for the nice KIP, it will allow other messaging use cases to >> be >>>>>>> onboarded to Kafka. >>>>>>> >>>>>>> +1 from me. >>>>>>> >>>>>>> Satish. >>>>>>> >>>>>>> On Tue, 7 May 2024 at 03:41, Jun Rao <j...@confluent.io.invalid> >> wrote: >>>>>>>> >>>>>>>> Hi, Andrew, >>>>>>>> >>>>>>>> Thanks for the KIP. +1 >>>>>>>> >>>>>>>> Jun >>>>>>>> >>>>>>>> On Mon, Mar 18, 2024 at 11:00 AM Edoardo Comar < >> edoardli...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Thanks Andrew, >>>>>>>>> >>>>>>>>> +1 (binding) >>>>>>>>> >>>>>>>>> Edo >>>>>>>>> >>>>>>>>> On Mon, 18 Mar 2024 at 16:32, Kenneth Eversole >>>>>>>>> <kevers...@cloudflare.com.invalid> wrote: >>>>>>>>>> >>>>>>>>>> Hi Andrew >>>>>>>>>> >>>>>>>>>> + 1 (Non-Binding) >>>>>>>>>> >>>>>>>>>> This will be great addition to Kafka >>>>>>>>>> >>>>>>>>>> On Mon, Mar 18, 2024 at 8:27 AM Apoorv Mittal < >>>>> apoorvmitta...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Andrew, >>>>>>>>>>> Thanks for writing the KIP. This is indeed going to be a valuable >>>>>>>>> addition >>>>>>>>>>> to the Kafka, excited to see the KIP. >>>>>>>>>>> >>>>>>>>>>> + 1 (Non-Binding) >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Apoorv Mittal >>>>>>>>>>> +44 7721681581 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Sun, Mar 17, 2024 at 11:16 PM Andrew Schofield < >>>>>>>>>>> andrew_schofield_j...@outlook.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> I’ve been working to complete KIP-932 over the past few months >> and >>>>>>>>>>>> discussions have quietened down. >>>>>>>>>>>> >>>>>>>>>>>> I’d like to open the voting for KIP-932: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-932%3A+Queues+for+Kafka >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Andrew
