Hi all, thank you so much for the discussion LB1: Yes. Custom TaskAssignor loading does the same as the consumer side. Each entry in group.streams.assignors is resolved to an instance at broker startup, and if it implements Configurable, the broker calls configure(config.originals()) on it. So a custom assignor can read broker‑level configuration the same way a consumer assignor can. Per‑group dynamic settings (e.g. num.standby.replicas) remain a separate channel via GroupSpec.configs(). I'll make this explicit in the KIP.
MJS1:Agreed. I'll mark all newly public-facing interfaces, classes, and methods as @InterfaceStability.Evolving so we can refine them in minor releases without a compatibility commitment. I'll update the KIP to state this for the whole new API surface. MJS2: Thank you for pointing it out. I will drop the warmup tasks from the output MemberAssignment Thanks. Gabriella On Sun, Jun 21, 2026 at 7:03 PM Matthias J. Sax <[email protected]> wrote: > Thanks for the KIP Gabby. > > Two comments about the newly added interfaces/records. > > MJS1: I would like to mark all new public facing API as @Evolving for > now. We might not get it right with the first release, and marking as > evolving would indicate that the API is not stable yet, and we can > introduce breaking changes in minor releases, allowing us to fix errors > quickly in future minor releases. > > MJS2: The existing (non public) assignor API returns `GroupAssignment` > which includes warmup task. I believe it's an artifact from the > "classic" case, in which the HA assignor was required to compute > warmups. However, with "streams" we can simplify the assignor and it > would only compute active and standby tasks, while only the GC (ie, > reconciler) will use warmup tasks. Thus, we should change the interface > accordingly, dropping warmup tasks from `MemberAssignment`. > > > About DJ_02: I have no experience with evolving Java `records` but it > seems they are more similar to `classes`. In general, `interfaces` > provide much higher flexibility than `classes`, so I agree with David > that it might be good to _only_ use `interfaces` and no `records`. > > > About the configs (DJ_01): I guess I am fine either way, to either > follow the current proposal, or mimic KIP-848 more closely with a single > broker config containing a mix of built-in assignor-names and > fully-qualified class names. > > In doubt I would agree with Lucas: the KIP-848 approach is a little bit > cleaner, as having one config seems easier than having two. Let's hear > from David about it. > > > > -Matthias > > > > On 6/16/26 1:55 AM, Lucas Brutschy via dev wrote: > > Hi all, > > > > DJ_01: Agree, and I'd push it a little further. On the consumer side > > the short name isn't configured at all — group.consumer.assignors is a > > single list of built-in names and class names, each entry is resolved > > to an instance, and they're mapped by name(); the first entry is the > > cluster default. So once the streams assignors are loaded, their short > > names already come from TaskAssignor.name(), which makes a separate > > group.streams.assignors.names redundant — a single > > group.streams.assignors list (default [sticky]) reproduces the > > consumer behavior. That also runs against the "mixed name/class > > configuration" rejected alternative, since the consumer config is > > exactly that mixed list. > > > > DJ_04: I agree with Gabby; "validate all dynamic group configs on the > > broker" seems out of scope for this KIP. > > > > LB1: When consumer assignors are loaded from group.consumer.assignors, > > instances implementing Configurable get the broker configs passed to > > them. Will the custom TaskAssignor loading do the same, so a custom > > assignor can read broker-level configuration the way a consumer > > assignor can? > > > > Thanks, > > Lucas > > > > On Fri, Jun 12, 2026 at 9:53 PM Gabriella Fu via dev > > <[email protected]> wrote: > >> > >> Hi David, > >> > >> Thanks a lot for the detailed review! > >> > >> *DJ_01:* That's a fair point. I'll update the KIP to use > >> group.streams.assignors to align with KIP-848 and remove the misleading > >> motivation section. > >> > >> *DJ_02:* Thanks for the suggestion. I went through the KIP-848 > interfaces > >> in org.apache.kafka.coordinator.group.api.assignor as well as the > existing > >> implementations (UniformAssignor, RangeAssignor), and the pattern makes > >> total sense now. I'll update the input types in the KIP accordingly: > >> > >> - > >> > >> GroupSpec will be changed to an interface with per-member accessors, > >> instead of returning a raw Map<String, AssignmentMemberSpec>. > >> - > >> > >> AssignmentMemberSpec will be replaced by two distinct interfaces: > >> MemberSubscription and MemberAssignment. This will separate the > member's > >> static metadata (processId, instanceId, rackId, clientTags) from its > >> runtime task assignment (active/standby/warmup tasks), mirroring the > >> clean split in KIP-848. > >> > >> *DJ_03:* Thank you for the suggestion. I'll rename it to configs() and > >> clarify in the text that it will only return configurations relevant to > the > >> assignment. > >> > >> *DJ_04:*Moving all dynamic group config validation to the broker makes > >> sense, but that would be a bit beyond the scope of this KIP. In this > case, > >> maybe we should just keep the config like this? > >> > >> Best, > >> Gabriella > >> > >> On Fri, Jun 12, 2026 at 10:54 AM David Jacot <[email protected]> wrote: > >> > >>> Hi Gabriella, > >>> > >>> Thanks for the KIP. I have a few high level comments: > >>> > >>> DJ_01: The motivation to not use `group.streams.assignors` and follow > >>> the pattern introduced by KIP-848 is pretty weak in my opinion, > >>> especially the "avoids leaking implementation class names into > >>> per-group dynamic configuration" part. The class names wont leak into > >>> the group config as the group config requires the name of the > >>> assignor. From a user perspective, it is exactly the same concept so > >>> using a different way to express it is wrong in my opinion. Should we > >>> just use `group.streams.assignors` to be consistent in our configs? > >>> > >>> DJ_02: I would suggest reconsidering the interface of the assignor. We > >>> started with a similar interface in KIP-848 and we realized during the > >>> implementation that using POJOS and returning Maps (e.g. for > >>> members()) was really inflexible. Moreover, I am not sure if using > >>> records is good from an evolutionary point of view. Using interfaces > >>> may be better. It also allows us to wrap internal objects to expose > >>> them to the assignor. Have you looked into the KIP-848's interface and > >>> the various assignors? > >>> > >>> DJ_03: Regarding the `assignmentConfigs()`, should we call it > >>> `configs()` as it is already clear that it is for the assignment. > >>> Moreover, I wonder if it is going to return all group configs or only > >>> a subset of them. Could you please clarify in the KIP? > >>> > >>> DJ_04: The dynamic group configs validation makes sense to me. Note > >>> that this pattern is not used today. I also wonder whether we should > >>> move the validation of all dynamic group configs there. To be > >>> discussed. > >>> > >>> Best, > >>> David > >>> > >>> On Thu, Jun 11, 2026 at 8:37 PM Gabriella Fu via dev > >>> <[email protected]> wrote: > >>>> > >>>> Hi all, > >>>> > >>>> I’d like to start the discussion for KIP-1357: Add broker side custom > >>>> assignors for "streams" groups > >>>> > >>>> KIP: > >>>> > >>> > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=430408758__;!!Ayb5sqE7!t7TDJ8XzjCLooFy9FhJLiHKHu5c7RJf8-41tQuqRykmMLMB2yl4ApQsRMSEVKCIIl0FuvbznfZcDww$ > >>>> JIRA: > >>> > https://urldefense.com/v3/__https://issues.apache.org/jira/browse/KAFKA-20683__;!!Ayb5sqE7!t7TDJ8XzjCLooFy9FhJLiHKHu5c7RJf8-41tQuqRykmMLMB2yl4ApQsRMSEVKCIIl0Fuvbzx-7tAbA$ > >>>> > >>>> Summary: > >>>> The Streams Rebalance Protocol (KIP-1071) moves task assignment from > the > >>>> client to the broker, but unlike the classic protocol and KIP-848 > >>> consumer > >>>> groups, it offers no way to plug in a custom assignor. This KIP closes > >>> that > >>>> gap by making the existing streams task assignor interfaces public API > >>> and > >>>> adding broker and group configurations so operators can load custom > >>>> assignor implementations and select them per group by short name, > with no > >>>> client-side involvement. > >>>> > >>>> Please let me know your feedback > >>>> > >>>> Thanks, > >>>> Gabriella Fu > >>> > >
