Thanks Ismael and Bill. It seems that nobody objects with the proposal. Hence, I prepared the following PR to update the upgrade notes.
- https://github.com/apache/kafka/pull/7363 (trunk and 2.3) - https://github.com/apache/kafka/pull/7364 (2.2) - https://github.com/apache/kafka-site/pull/229 Will also revert the cherry-pick commit in 2.1 branch and update the Jira ticket. -Matthias On 9/16/19 4:22 PM, Ismael Juma wrote: > +1 to the proposal. Let's also highlight this in the release notes for > 2.2.2 and 2.3.0 please. > > Ismael > > On Wed, Sep 11, 2019 at 10:23 AM Matthias J. Sax <matth...@confluent.io> > wrote: > >> Hi, >> >> recently a user reported an issue upgrading a Kafka Streams application >> from 2.2.0 to 2.2.1 (cf >> https://mail-archives.apache.org/mod_mbox/kafka-users/201908.mbox/ >> <CAG8RrxkDAhCrGQ1%3DpBpX-Jcxagk_xkmRk4MBQeZYAP6ZGHoK_w%40mail.gmail.com>) >> >> After some investigation, we identified >> https://issues.apache.org/jira/browse/KAFKA-7895 to be the root cause of >> the problem. >> >> The fix for KAFKA-7895 is using message headers and thus requires broker >> version 0.11.0 (or newer) and message format 0.11 (or newer). Hence, >> while a Kafka Streams application version 2.2.0 is compatible to older >> brokers (0.10.1 and 0.10.2) and only requires message format 0.10, the >> backward compatibility was broken accidentally in 2.2.1. >> >> The fix is also contained in 2.3.0 release and cherry-picked to 2.1 >> branch (2.1.2 is not released yet, and thus 2.1 users are not affected >> as this point). >> >> Note: only users that use `suppress()` operator in their program are >> affected. >> >> We should not break streams-broker backward compatibility in bug-fix >> releases at all and avoid for minor releases. However, it seems >> necessary to have the fix in 2.3.0 though -- otherwise, `suppress()` is >> effectively useless and it does not seem to be a good idea to fix the >> bug only in the next major release. Hence, trading-off some backward >> compatibility in a minor release seems to be acceptable for this case, >> considering that 0.11.0 was release 2 years ago. >> >> For 2.2.1, it is more challenging to decide how to move forward, because >> we should not have broken streams-broker compatibility but 2.2.1 is >> already released and we can only react after the fact. >> >> From my point of view, the best way is to keep the fix and update the >> release notes and documentation accordingly. The main reason for my >> suggestions is that we would expect a majority of users to be on 0.11.0 >> brokers already and the fix will work for them -- reverting the fix in >> 2.2.2 seems to be worse for all those users on newer broker versions. We >> also know that `suppress()` is a highly demanded feature and a lot of >> people waited for a fix. >> >> The expected minority of users that are on 0.10.1 / 0.10.2 brokers, or >> newer brokers but still on message format 0.10 would either need to stay >> on Kafka Streams 2.2.0 or upgrade their brokers/message format >> accordingly. However, upgrading brokers/message format is de-facto >> already required for 2.2.1 and thus keeping the fix would not make the >> situation worse. >> >> For 2.1, I would suggest to revert the fix to make sure we don't break >> streams-broker compatibility for 2.1.x users. If those users need the >> fix for `suppress()` they need to upgrade to 2.2.1/2.3.0 or newer and >> make sure their brokers are on 0.11.0 with message format 0.11, too. >> >> >> TL;DR; the proposal is: >> >> (1) revert the fix for KAFKA-7895 in 2.1 branch >> (2) keep the fix for KAFKA-7895 in 2.2.1 and 2.3.0 >> >> Impact: >> >> - Kafka Streams 2.1.x and 2.2.0 applications are backward compatible >> back to 0.10.1 brokers, requiring message format 0.10 >> - Kafka Streams 2.2.2 / 2.3.0 application (or newer) are backward >> compatible back to 0.11.0 brokers, requiring message format 0.11 >> >> >> Please let us know what you think about this proposal. >> >> >> -Matthias >> >> >> >> >
signature.asc
Description: OpenPGP digital signature
