[
https://issues.apache.org/jira/browse/KAFKA-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16880079#comment-16880079
]
Martin Kamp Jensen commented on KAFKA-7563:
-------------------------------------------
After upgrading to 2.3.0 clients (and currently 2.2.1 broker) we have not
reproduced this error in a couple of 100 runs.
> Single broker sends incorrect metadata for topic partitions
> -----------------------------------------------------------
>
> Key: KAFKA-7563
> URL: https://issues.apache.org/jira/browse/KAFKA-7563
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 2.0.0
> Reporter: Martin Kamp Jensen
> Priority: Major
> Attachments: kafka.log, zookeeper.log
>
>
> When starting our Kafka Streams application in a test setup with just one
> Kafka broker we are seeing the following error roughly 1 out of 15 runs:
> {{StreamsException: Existing internal topic
> alarm-message-streams-alarm-from-unknown-asset-changelog has invalid
> partitions: expected: 32; actual: 25. Use 'kafka.tools.StreamsResetter' tool
> to clean up invalid topics before processing.}}
> (Note: It is not always the same topic that causes the error.)
> When we see the error above the actual number of partitions varies (expected
> is 32, actual is above 0 and below 32).
> Before each test run the Kafka broker is started without data (using
> [https://hub.docker.com/r/wurstmeister/kafka/]).
> We have never seen this happen in non-test where we are running with 6 Kafka
> brokers. However, we are running a significantly higher number of test runs
> than deploys to non-test.
> After some investigation (including using AdminClient to describe the topics
> when the Kafka Streams application got the StreamsException and confirming
> that AdminClient also reports that a topic has the wrong number of
> partitions!) we implemented the following workaround: When the Kafka Streams
> application fails with the exception, we stop the application, stop the Kafka
> broker, start the Kafka broker, and finally start the application. Then the
> exception is not thrown. Of course this does not explain or fix the real
> issue at hand but it is still important because we all hate flaky tests.
> Kafka and ZooKeeper log files from a run where the exception above occurred
> and where applying the workaround described above enabled us to continue
> without the exception are attached.
> This issue was created by request of Matthias J. Sax at
> https://stackoverflow.com/questions/52943653/existing-internal-topic-has-invalid-partitions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
