Hi Artem, Thanks for the update. I have some questions about it:
1. Could you explain why you need the `partitioner` return -1? In which case we need it? And how it is used in your KIP? 2. What does the "partitioner.sticky.batch.size" mean? In the "Configuration" part, you didn't explain it. And default to 0, I guess it's the same as current behavior for backward compatibility, right? You should mention it. 3. I'm thinking we can have a threshold to the "partitioner.sticky.batch.size". Let's say, we already accumulate 15.5KB in partition1, and sent. So when next batch created, in your current design, we still stick to partition1, until 16KB reached, and then we create a new batch to change to next partition, ex: partition2. But I think if we set a threshold to 95% (for example), we can know previous 15.5KB already exceeds the threshold so that we can directly create new batch for next records. This way should be able to make it more efficient. WDYT? 4. I think the improved queuing logic should be good enough. I can't get the benefit of having `partition.availability.timeout.ms` config. In short, you want to make the partitioner take the broker load into consideration. We can just improve that in the queuing logic (and you already did it). Why should we add the config? Could you use some examples to explain why we need it. Thank you. Luke On Wed, Feb 16, 2022 at 8:57 AM Artem Livshits <alivsh...@confluent.io.invalid> wrote: > Hello, > > Please add your comments about the KIP. If there are no considerations, > I'll put it up for vote in the next few days. > > -Artem > > On Mon, Feb 7, 2022 at 6:01 PM Artem Livshits <alivsh...@confluent.io> > wrote: > > > Hello, > > > > After trying a few prototypes, I've made some changes to the public > > interface. Please see the updated document > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+Sticky+Partitioner > > . > > > > -Artem > > > > On Thu, Nov 4, 2021 at 10:37 AM Artem Livshits <alivsh...@confluent.io> > > wrote: > > > >> Hello, > >> > >> This is the discussion thread for > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+Sticky+Partitioner > >> . > >> > >> The proposal is a bug fix for > >> https://issues.apache.org/jira/browse/KAFKA-10888, but it does include > a > >> client config change, therefore we have a KIP to discuss. > >> > >> -Artem > >> > > >
