Thanks for addressing this Matthias. I'm +1 for the proposal as well. -Bill
On Mon, Sep 16, 2019 at 4:23 PM Ismael Juma <ism...@juma.me.uk> wrote: > +1 to the proposal. Let's also highlight this in the release notes for > 2.2.2 and 2.3.0 please. > > Ismael > > On Wed, Sep 11, 2019 at 10:23 AM Matthias J. Sax <matth...@confluent.io> > wrote: > > > Hi, > > > > recently a user reported an issue upgrading a Kafka Streams application > > from 2.2.0 to 2.2.1 (cf > > https://mail-archives.apache.org/mod_mbox/kafka-users/201908.mbox/ > > <CAG8RrxkDAhCrGQ1%3DpBpX-Jcxagk_xkmRk4MBQeZYAP6ZGHoK_w%40mail.gmail.com > >) > > > > After some investigation, we identified > > https://issues.apache.org/jira/browse/KAFKA-7895 to be the root cause of > > the problem. > > > > The fix for KAFKA-7895 is using message headers and thus requires broker > > version 0.11.0 (or newer) and message format 0.11 (or newer). Hence, > > while a Kafka Streams application version 2.2.0 is compatible to older > > brokers (0.10.1 and 0.10.2) and only requires message format 0.10, the > > backward compatibility was broken accidentally in 2.2.1. > > > > The fix is also contained in 2.3.0 release and cherry-picked to 2.1 > > branch (2.1.2 is not released yet, and thus 2.1 users are not affected > > as this point). > > > > Note: only users that use `suppress()` operator in their program are > > affected. > > > > We should not break streams-broker backward compatibility in bug-fix > > releases at all and avoid for minor releases. However, it seems > > necessary to have the fix in 2.3.0 though -- otherwise, `suppress()` is > > effectively useless and it does not seem to be a good idea to fix the > > bug only in the next major release. Hence, trading-off some backward > > compatibility in a minor release seems to be acceptable for this case, > > considering that 0.11.0 was release 2 years ago. > > > > For 2.2.1, it is more challenging to decide how to move forward, because > > we should not have broken streams-broker compatibility but 2.2.1 is > > already released and we can only react after the fact. > > > > From my point of view, the best way is to keep the fix and update the > > release notes and documentation accordingly. The main reason for my > > suggestions is that we would expect a majority of users to be on 0.11.0 > > brokers already and the fix will work for them -- reverting the fix in > > 2.2.2 seems to be worse for all those users on newer broker versions. We > > also know that `suppress()` is a highly demanded feature and a lot of > > people waited for a fix. > > > > The expected minority of users that are on 0.10.1 / 0.10.2 brokers, or > > newer brokers but still on message format 0.10 would either need to stay > > on Kafka Streams 2.2.0 or upgrade their brokers/message format > > accordingly. However, upgrading brokers/message format is de-facto > > already required for 2.2.1 and thus keeping the fix would not make the > > situation worse. > > > > For 2.1, I would suggest to revert the fix to make sure we don't break > > streams-broker compatibility for 2.1.x users. If those users need the > > fix for `suppress()` they need to upgrade to 2.2.1/2.3.0 or newer and > > make sure their brokers are on 0.11.0 with message format 0.11, too. > > > > > > TL;DR; the proposal is: > > > > (1) revert the fix for KAFKA-7895 in 2.1 branch > > (2) keep the fix for KAFKA-7895 in 2.2.1 and 2.3.0 > > > > Impact: > > > > - Kafka Streams 2.1.x and 2.2.0 applications are backward compatible > > back to 0.10.1 brokers, requiring message format 0.10 > > - Kafka Streams 2.2.2 / 2.3.0 application (or newer) are backward > > compatible back to 0.11.0 brokers, requiring message format 0.11 > > > > > > Please let us know what you think about this proposal. > > > > > > -Matthias > > > > > > > > >
