[
https://issues.apache.org/jira/browse/KAFKA-7190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588370#comment-16588370
]
Guozhang Wang commented on KAFKA-7190:
--------------------------------------
[~lambdaliu] As Jason's proposal #5 mentioned, we do plan to keep PID in the
cache until it was expired, instead of immediately delete when the last record
is deleted. However, even in this case, when there is a leader migration there
are still inconsistent PID caching, because only leader maintains PID cache. So
think about this sequence:
1. last record of producer deleted on leader.
2. deletion migrated to follower, which delete the record as well.
3. leader migration happens, the follower becomes the new leader and builds the
PID cache from logs, which do not have the producer ids.
4. producer sends to the new leader, which does not recognize it any more.
Hence, the UNKNOWN_PRODUCER_ID can still be sent back. The rationale is that
since it is quite rare, using the safer way to bump up the epoch (which is more
costly than resetting sequence number) is fine.
> Under low traffic conditions purging repartition topics cause WARN statements
> about UNKNOWN_PRODUCER_ID
> ---------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-7190
> URL: https://issues.apache.org/jira/browse/KAFKA-7190
> Project: Kafka
> Issue Type: Improvement
> Components: core, streams
> Affects Versions: 1.1.0, 1.1.1
> Reporter: Bill Bejeck
> Assignee: lambdaliu
> Priority: Major
>
> When a streams application has little traffic, then it is possible that
> consumer purging would delete
> even the last message sent by a producer (i.e., all the messages sent by
> this producer have been consumed and committed), and as a result, the broker
> would delete that producer's ID. The next time when this producer tries to
> send, it will get this UNKNOWN_PRODUCER_ID error code, but in this case,
> this error is retriable: the producer would just get a new producer id and
> retries, and then this time it will succeed.
>
> Possible fixes could be on the broker side, i.e., delaying the deletion of
> the produderIDs for a more extended period or on the streams side developing
> a more conservative approach to deleting offsets from repartition topics
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
