Thanks for prompt response. I am not sure I understand correctly, but I am still confused why switching inter.broker.protocol.version in the last step would make the process irreversible. If we agree that log conversion to a new format is applied when a new value of log.message.format.version or broker's default one is applied then according to documentation switching back to old version of broker will make no harm. So either a broker is able to read a newer versions of log and dismiss parts he doesn’t understand or the conversion itself occurs only at certain combination of inter.broker.protocol.version and log.message.format.version. Or maybe I am completely wrong with my assumptions.
Regards, Artur -----Original Message----- From: M. Manna [mailto:manme...@gmail.com] Sent: Thursday, May 09, 2019 12:19 PM To: Kafka Users <users@kafka.apache.org> Subject: Re: Kafka upgrade process details Sabre Email Notification: This email is from an EXTERNAL source. Please use caution when clicking on links or opening attachments from an unknown or suspicious sender.To report a suspected phishing email, browse to: help.sabre.com > Risk & Security > Report a Security Issue ______________________________________________________________________ Artur, The upgrade process is such that 1) You ensure that there is a hard-check on protocol version if not exists already. As you have already mentioned above, in #3 - it's to ensure that min verson for msg formats are being adhered to before upgrade. 2) broker protocol version is to ensure that when you do rolling upgrade there is a minimum compatibility between inter-broker comms (if that makes sense). API versions are there for a specific reason (to ensure compatbility is maintained first before the protocol/msg formats are fixed). Also, it ensures that a regular upgrade (e.g. from bugfix->major release) happens seamlessly via rolling restart. If you want to get techincal about it, you can look at kafka Table load and config load in github codebase. Once you've initiated a rolling restart by setting inter.broker.protocl.version and log.messge.format.version there is no way back - but i am happy to be proven wrong as I have only done rolling restart successfully (never needed to go back :) ) On Thu, 9 May 2019 at 09:54, Chylek, Artur <artur.chy...@sabre.com> wrote: > Hi, > I read the documentation about upgrading Kafka( > https://urldefense.proofpoint.com/v2/url?u=http-3A__kafka.apache.org_21_documentation.html-23upgrade-5F2-5F1-5F0&d=DwIBaQ&c=FXJfUb1oWgygD0uNz-ujnA&r=bVI5M04li2b_AW9E6XWAZb5H4NuzOzdzPeKTA_sjdMg&m=3g5HRSTfPHijn_jhhzYIB88jScofj68jB1AmrcMPBC0&s=qiYRoeea_IxCjRGx1Vi7ylwk2vuCiYGGoPhhGYmiwos&e=) > but I have > questions that I believe the documentation doesn't cover. I am planning to > upgrade Kafka from 2.0.0 to 2.1.0 and would like to make sure what to do > when something goes wrong - I am mostly interested in reverting back to the > older version. I admit that I am confused about API versions, log message > versions and inter broker protocol versions. > > According to the documentation these are the steps I need to follow: > > 1. Set inter.broker.protocol.version to 2.0.0 > 2. I don't have currently log.message.format.version set to 2.0.0, so > according to documentation I don't need to specify it > 3. Upgrade broker code and restart it. This is I am not sure what > happens next. Since I don't have log.message.format.version set then > broker's default value will be used - I guess 2.1.1. Does it mean that: > * New messages that arrive from producers will be saved in a log > file with a new format? > * Old messages that already exist in log files will be converted to > the new format once the broker starts? > * Does inter.broker.protocol.version has anything to do with > log.message.format.version? > I guess a) and b) are not true since according to the documentation up to > this point I can revert back to the older version of Kafka. Could you > confirm? > > 1. Change inter.broker.protocol.version to 2.1.1 and restart brokers - > after this is done reverting back is not possible. Could you explain what > is happening in Kafka so that reverting cannot be done? > > Regards, > Artur >
