Oh yes! You can run a script which checks that the topic IDs in Zk are
consistent with the ones that are available on partition.metadata.

An example script would look something like this:
https://gist.github.com/divijvaidya/8a940a3d67bc8b215c95da54f5eff961

I will also add this script to the JIRA I linked earlier. Please add a +1
to the JIRA since it would help measure the impact of this bug and hence,
will help drive prioritization of a fix in the community.

--
Divij Vaidya

On Mon, Dec 19, 2022 at 1:23 PM Atul Kumar (atkumar3)
<atul.ku...@appdynamics.com.invalid> wrote:

> Hi Divij,
>
> As this issue manifests after the Kafka controller restart,  is there a
> way to detect this corrupted Kafka cluster before restarting the controller?
>
>
>
>
>
> Regards,​
> Atul
> ________________________________
> From: Atul Kumar (atkumar3) <atul.ku...@appdynamics.com>
> Sent: 16 December 2022 18:24
> To: atul.ku...@appdynamics.com.invalid <atul.ku...@appdynamics.com.INVALID>;
> users@kafka.apache.org <users@kafka.apache.org>
> Subject: Re: [SUSPECTED SPAM] Re: Critical bug in Kafka 2.8.1 | topic Id
> in memory: <> does not match the topic Id for partition <> provided in the
> request: <>.
>
> Hi Divij,
>
> Thanks for the reply!
>
> We were using the Kafka-manager tool (with an older Kafka client) which
> was the root cause of this issue.
>
>
>
>
> Regards,​
> Atul
> ________________________________
> From: Divij Vaidya <divijvaidy...@gmail.com>
> Sent: 08 December 2022 17:55
> To: atul.ku...@appdynamics.com.invalid <atul.ku...@appdynamics.com
> .invalid>
> Cc: users@kafka.apache.org <users@kafka.apache.org>
> Subject: [SUSPECTED SPAM] Re: Critical bug in Kafka 2.8.1 | topic Id in
> memory: <> does not match the topic Id for partition <> provided in the
> request: <>.
>
> Hi Atul
>
> There is a known bug which has similar symptoms that you observed. See:
> https://issues.apache.org/jira/browse/KAFKA-14190
> The bug manifests when you use a client < 2.8.0 with `--zookeeper` flag. It
> could be avoided by either upgrading the client or by using
> `--bootstrap-server` instead of `--zookeeper`.
>
> Does the above explanation match your Kafka setup?
>
> --
> Divij Vaidya
>
>
>
> On Thu, Dec 8, 2022 at 1:16 PM Atul Kumar (atkumar3)
> <atul.ku...@appdynamics.com.invalid> wrote:
>
> > Hello everyone,
> >
> > We are using Kafka 2.8.1, and recently while rolling restart of one of
> our
> > clusters, we encountered the following issue on the Kafka cluster
> >
> > topic Id in memory: <> does not match the topic Id for partition <>
> > provided in the request: <>.
> >
> > All the produce requests to the impacted topic were failing. We were able
> > to fix the issue by deleting partition.metadata from each broker for the
> > impacted topic and doing a rolling restart of the Kafka cluster. We are
> > using the Kafka stream application.
> >
> > The same issue was reported in Jira:
> > https://issues.apache.org/jira/browse/KAFKA-12835  and is marked fixed
> > with version 2.8.1 but we can still see this issue.
> >
> > Any help would be highly appreciated on any fix for this issue.
> >
> > Thanks,
> > Atul Kumar
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Regards,​
> > Atul
> >
>

Reply via email to