Hey IJ, Thanks for your interest in the KIP.
My point was simply that the round-robin should happen even if the key is not null. As for the importance of key in our case, we treat the key as metadata. Each key is composed of certain info which are parsed by our consumer thread. We will then determine whether it's an actionable message (e.g. process it), or a loopback(ignore it). You could argue, "Why not append this metadata with the record and parse it there?". But that means the following: 1) I'm always passing null key to achieve this - I would like to pass Null/Not-Null/Other key i.e. flexibility 2) Suppose the message size is 99 KB and and max message bytes allowed is 100K. Now prefixing metadata with message results into the actual message being 101K. This will fail at producer level and cause a retry/log this in our DB for future pickup. To avoid all these, we are simply proposing this new partitioner class. but all Kafka new releases will still have DefaultPartitioner as default, unless they change the prop file to use our new class. Regards, On Sun, 21 Oct 2018 at 04:05, Ismael Juma <ism...@juma.me.uk> wrote: > Thanks for the KIP. Can you please elaborate on the need for the key in > this case? The KIP simply states that the key is needed for metadata, but > doesn't give any more details. > > Ismael > > On Tue, Sep 4, 2018 at 3:39 AM M. Manna <manme...@gmail.com> wrote: > > > Hello, > > > > I have made necessary changes as per the original discussion thread, and > > would like to put it for votes. > > > > Thank you very much for your suggestion and guidance so far. > > > > Regards, > > >
