[
https://issues.apache.org/jira/browse/KAFKA-12835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555852#comment-17555852
]
ZhenChun Pan commented on KAFKA-12835:
--------------------------------------
[~jolshan] Hello, I find an error similar to this issue mentioned below when I
update my kafka from 2.7.0 to 2.8.1.
[2022-06-17 09:34:56,599] ERROR [Broker id=0] Topic Id in memory:
PV_7diK7RJaS45KsEH4sng does not match the topic Id for partition
__consumer_offsets-14 provided in the request: X_hrEUetShKyQ_RR_pEo9A.
(state.change.logger)
I found that the topic id of __consumer_offsets in partition.metadata is
PV_7diK7RJaS45KsEH4sng. And in zookeeper the topic id is X_hrEUetShKyQ_RR_pEo9A.
Then I delete all partition.metadata of topic __consumer_offsets, restart
kafka, and fix this probelm.
I try to recurrence this probelm the second time, but failed.
I want to ask in which situation will make the topic id of __consumer_offsets
diffirent between partition.metadata and zookeeper. Because when topic
__consumer_offsets have this problem, the consumer will not work completely. I
had to make attention on it.
> Topic IDs can mismatch on brokers (after interbroker protocol version update)
> -----------------------------------------------------------------------------
>
> Key: KAFKA-12835
> URL: https://issues.apache.org/jira/browse/KAFKA-12835
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 2.8.0
> Reporter: Ivan Yurchenko
> Assignee: Justine Olshan
> Priority: Major
> Fix For: 2.8.1, 3.0.0
>
>
> We had a Kafka cluster running 2.8 version with interbroker protocol set to
> 2.7. It had a number of topics and everything was fine.
> Then we decided to update the interbroker protocol to 2.8 by the following
> procedure:
> 1. Run new brokers with the interbroker protocol set to 2.8.
> 2. Move the data from the old brokers to the new ones (normal partition
> reassignment API).
> 3. Decommission the old brokers.
> At the stage 2 we had the problem: old brokers started failing on
> {{LeaderAndIsrRequest}} handling with
> {code:java}
> ERROR [Broker id=<...>] Topic Id in memory: <...> does not match the topic Id
> for partition <...> provided in the request: <...>. (state.change.logger)
> {code}
> for multiple topics. Topics were not recreated.
> We checked {{partition.metadata}} files and IDs there were indeed different
> from the values in ZooKeeper. It was fixed by deleting the metadata files
> (and letting them be recreated).
>
> The logs, unfortunately, didn't show anything that might point to the cause
> of the issue (or it happened longer ago than we store the logs).
> We tried to reproduce this also, but no success.
> If the community can point out what to check or beware of in future, it will
> be great. We'll be happy to provide additional information if needed. Thank
> you!
> Sorry for the ticket that might be not very actionable. We hope to at least
> rise awareness of this issue.
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
