Hi Israel,

I followed these steps but instead of setting inter.broker.protocol.version
to 2.8 in step 3 I just removed it. Dunno if that changes anything ?
Regarding zookeeper we're using 3.5.8
Luckily this was a pre-production cluster, we were testing the upgrade here
and we do have backups.

Anyway to ensute that all zk nodes are in sync when performing the upgrade
?
I fail to understand how this can lead to such situation. Shouldn't the
unsync zk node data be discared automatically ?

Le mar. 25 janv. 2022 à 15:32, Israel Ekpo <israele...@gmail.com> a écrit :

> Hi Nicolas,
>
> Did you follow the upgrade steps here?
>
> https://kafka.apache.org/documentation/#upgrade_2_8_1
>
> Also, the recommended Zookeeper versions for 2.0 (3.4.13) is different from
> that of 2.8 (3.5.9)
>
> https://github.com/apache/kafka/blob/2.0/gradle/dependencies.gradle#L87
> https://github.com/apache/kafka/blob/2.8/gradle/dependencies.gradle#L122
>
> It looks like your Zookeeper quorum was not in sync before the upgrade. 2
> of them seem to agree with the topic id but 1 is different.
>
> If this is production, you may have to decide which one to keep and then
> you may need to upgrade Zookeeper first make sure they are all in sync with
> your approved data, then you can bring Kafka online.
>
> I generally recommend to back up your cluster data (in remote storage) and
> config data in version control before any upgrades so that you can always
> restore the state if all does not go well.
>
> This could be due to Zookeeper version incompatibility or bad state of data
> in Zookeeper before the upgrade.
>
> Israel Ekpo
> Lead Instructor, IzzyAcademy.com
> https://www.youtube.com/c/izzyacademy
> https://izzyacademy.com/
>
>
> On Tue, Jan 25, 2022 at 7:21 AM Nicolas Carlot
> <nicolas.car...@chronopost.fr.invalid> wrote:
>
> > Hello everyone,
> >
> > I just had a major failure while upgrading a kafka cluster from 2.0 to
> > 2.8.1 following the provided migration process.
> > I understand that a topicId is now given to each topic within zookeeper
> and
> > meta.properties of each partition.
> > While describing the topic, it seems I have different topicId depending
> on
> > the zk node i'm querying:
> >
> > [kafkaadm@lyn3e154(PFI):~ 13:16:28]$
> > /opt/java/j2ee/kafka/bin/kafka-topics.sh --zookeeper
> satezookeeperi1:62181
> > --describe --topic PARCEL360.LT
> > Topic: PARCEL360.LT     TopicId: XjIuCqy2TcKu-M5smrz9iA PartitionCount:
> 10
> >      ReplicationFactor: 3    Configs: compression.type=lz4
> >         Topic: PARCEL360.LT     Partition: 0    Leader: 3
>  Replicas:
> > 1,2,3 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 1    Leader: 3
>  Replicas:
> > 2,3,1 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 2    Leader: 3
>  Replicas:
> > 3,1,2 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 3    Leader: 3
>  Replicas:
> > 1,2,3 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 4    Leader: 3
>  Replicas:
> > 2,3,1 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 5    Leader: 3
>  Replicas:
> > 3,1,2 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 6    Leader: 3
>  Replicas:
> > 1,2,3 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 7    Leader: 3
>  Replicas:
> > 2,3,1 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 8    Leader: 3
>  Replicas:
> > 3,1,2 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 9    Leader: 3
>  Replicas:
> > 1,2,3 Isr: 3
> > [kafkaadm@lyn3e154(PFI):~ 13:17:06]$
> > /opt/java/j2ee/kafka/bin/kafka-topics.sh --zookeeper
> satezookeeperi2:62181
> > --describe --topic PARCEL360.LT
> > Topic: PARCEL360.LT     TopicId: zwbQDd9NRjGwq-v2twHfIQ PartitionCount:
> 10
> >      ReplicationFactor: 3    Configs: compression.type=lz4
> >         Topic: PARCEL360.LT     Partition: 0    Leader: 3
>  Replicas:
> > 1,2,3 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 1    Leader: 3
>  Replicas:
> > 2,3,1 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 2    Leader: 3
>  Replicas:
> > 3,1,2 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 3    Leader: 3
>  Replicas:
> > 1,2,3 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 4    Leader: 3
>  Replicas:
> > 2,3,1 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 5    Leader: 3
>  Replicas:
> > 3,1,2 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 6    Leader: 3
>  Replicas:
> > 1,2,3 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 7    Leader: 3
>  Replicas:
> > 2,3,1 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 8    Leader: 3
>  Replicas:
> > 3,1,2 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 9    Leader: 3
>  Replicas:
> > 1,2,3 Isr: 3
> > [kafkaadm@lyn3e154(PFI):~ 13:17:11]$
> > /opt/java/j2ee/kafka/bin/kafka-topics.sh --zookeeper
> satezookeeperi3:62181
> > --describe --topic PARCEL360.LT
> > Topic: PARCEL360.LT     TopicId: XjIuCqy2TcKu-M5smrz9iA PartitionCount:
> 10
> >      ReplicationFactor: 3    Configs: compression.type=lz4
> >         Topic: PARCEL360.LT     Partition: 0    Leader: 3
>  Replicas:
> > 1,2,3 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 1    Leader: 3
>  Replicas:
> > 2,3,1 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 2    Leader: 3
>  Replicas:
> > 3,1,2 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 3    Leader: 3
>  Replicas:
> > 1,2,3 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 4    Leader: 3
>  Replicas:
> > 2,3,1 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 5    Leader: 3
>  Replicas:
> > 3,1,2 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 6    Leader: 3
>  Replicas:
> > 1,2,3 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 7    Leader: 3
>  Replicas:
> > 2,3,1 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 8    Leader: 3
>  Replicas:
> > 3,1,2 Isr: 3
> >         Topic: PARCEL360.LT     Partition: 9    Leader: 3
>  Replicas:
> > 1,2,3 Isr: 3
> >
> >
> > Any idea what's happening here ?
> >
> >
> >
> >
> > --
> > <https://www.chronopost.fr/fr?xtatc=INT-149>
> >
> >
> > *Nicolas Carlot*
> > *Lead dev*Direction des Systèmes d'Information
> >
> >
> > 3 boulevard Romain Rolland
> > 75014 Paris
> > <https://mailsign.chronopost.fr/linkc/K0ppSHlnPT0-L3A4PQ>
> > <https://mailsign.chronopost.fr/linkc/K0ppSHlnPT0-L1pzPQ>
> > <https://mailsign.chronopost.fr/linkc/K0ppSHlnPT0-L1pRPQ>
> > <https://mailsign.chronopost.fr/linkc/K0ppSHlnPT0-L1pVPQ>
> >
>


-- 
<https://www.chronopost.fr/fr?xtatc=INT-149>


*Nicolas Carlot*
*Lead dev*Direction des Systèmes d'Information


3 boulevard Romain Rolland
75014 Paris
<https://mailsign.chronopost.fr/linkc/K0ppSHlnPT0-L3A4PQ>
<https://mailsign.chronopost.fr/linkc/K0ppSHlnPT0-L1pzPQ>
<https://mailsign.chronopost.fr/linkc/K0ppSHlnPT0-L1pRPQ>
<https://mailsign.chronopost.fr/linkc/K0ppSHlnPT0-L1pVPQ>

Reply via email to