+1 to the comment. > I still feel we are doing all of this only because of a few anti-pattern or misconfigured producers and not because we have “too many Producer”. I believe that implementing Producer heartbeat and remove short-lived PIDs from the cache if we didn’t receive heartbeat will be more simpler and step on right direction to improve idempotent logic and maybe try to make PID get reused between session which will implement a real idempotent producer instead of idempotent session. I admit this wouldn’t help with old clients but it will put us on the right path.
This issue is very complicated and I appreciate the attention on it. Hopefully we can find a good solution working together :) Justine On Wed, May 15, 2024 at 8:36 AM Omnia Ibrahim <o.g.h.ibra...@gmail.com> wrote: > Also in the rejection alternatives you listed an approved KIP which is a > bit confusing can you move this to motivations instead > > > On 15 May 2024, at 14:35, Claude Warren <cla...@apache.org> wrote: > > > > This is a proposal that should solve the OOM problem on the servers > without > > some of the other proposed KIPs being active. > > > > Full details in > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1044%3A+A+proposal+to+change+idempotent+producer+--+server+implementation > >
