Sure, we can compile a list for Kafka Streams. But the KIP would be for 3.0, so I don't think it's urgent to do it now?
-Matthias On 5/8/20 3:47 PM, Colin McCabe wrote: > Thanks, Guozhang-- sounds like a good plan. > > I think it would be good to have a list of deprecated streams APIs that we > want to remove in 3.0. Maybe it's easiest to do that as its own KIP? > > For MirrorMaker 1, we should have a KIP to deprecate its use in 2.6 if we > want to remove it in 3.0. I don't have a good sense of how practical it is > to deprecate this now, so I will defer to others here. But the KIP freeze > for 2.6 is coming soon, so if we want to make the case, now is the time. > > best, > Colin > > > On Thu, May 7, 2020, at 16:28, Guozhang Wang wrote: >> Hey folks, >> >> Sorry for stating that the bridge release would not break any compatibility >> before, which is incorrect and confused many people. >> >> I think one way to think about the versioning is that: >> >> 0) In a 2.x version moving ahead we would deprecate the ZK-dependent tools >> such as --zookeeper flags from various scripts (KIP-555) >> >> 1) In 3.0 we would at least make one incompatible change for example to >> remove the deprecated ZK flags. >> >> 2) In a future major version (e.g. 4.0) we would drop ZK entirely, >> including usages such as security credentials / broker registration / etc >> which are via ZK today as well. >> >> Then for the bridge release(s), it can be any or all of 3.x. >> >> >> For 1), I'd love to add a few more incompatibility changes in 3.0 from >> Kafka Streams: we evolve Streams public APIs by deprecating and then remove >> in major releases, and since 2.0 we've accumulated quite a few deprecated >> APIs, and I can compile a list of KIPs that contain those if people are >> interested. >> >> >> Guozhang >> >> >> On Thu, May 7, 2020 at 3:53 PM Colin McCabe <cmcc...@apache.org> wrote: >> >>> On Wed, May 6, 2020, at 21:33, Ryanne Dolan wrote: >>>>> In fact, we know that the bridge release will involve at least one >>>>> incompatible change. We will need to drop support for the --zookeeper >>>>> flags in the command-line tools. >>>> >>>> If the bridge release(s) and the subsequent post-ZK release are _both_ >>>> breaking changes, I think we only have one option: the 3.x line are the >>>> bridge release(s), and ZK is removed in 4.0, as suggested by Andrew >>>> Schofield. >>>> >>>> Specifically: >>>> - in order to _remove_ (not merely deprecate) the --zookeeper args, we >>> will >>>> need a major release. >>>> - in oder to drop support for ZK entirely (e.g. break a bunch of external >>>> tooling like Cruise Control), we will need a major release. >>>> >>>> I count two major releases. >>> >>> Hi Ryanne, >>> >>> I agree that dropping ZK completely will need a new major release after >>> 3.0. I think that's OK and in keeping with how we've handled deprecation >>> and removal in the past. It's important for users to have a smooth upgrade >>> path. >>> >>> best, >>> Colin >>> >>>> >>>> Ryanne >>>> >>>> - >>>> >>>> On Wed, May 6, 2020 at 10:52 PM Colin McCabe <cmcc...@apache.org> wrote: >>>> >>>>> On Mon, May 4, 2020, at 17:12, Ryanne Dolan wrote: >>>>>> Hey Colin, I think we should wait until after KIP-500's "bridge >>>>>> release" so there is a clean break from Zookeeper after 3.0. The >>>>>> bridge release by definition is an attempt to not break anything, so >>>>>> it theoretically doesn't warrant a major release. >>>>> >>>>> Hi Ryanne, >>>>> >>>>> I think it's important to clarify this a little bit. The bridge >>> release >>>>> (really, releases, plural) allow you to upgrade from a cluster that is >>>>> using ZooKeeper to one that is not using ZooKeeper. But, that doesn't >>>>> imply that the bridge release itself doesn't break anything. Upgrading >>>>> to the bridge release itself might involve some minor incompatibility. >>>>> >>>>> Kafka does occasionally have incompatible changes. In those cases, we >>>>> bump the major version number. One example is that when we went from >>>>> Kafka 1.x to Kafka 2.0, we dropped support for JDK7. This is an >>>>> incompatible change. >>>>> >>>>> In fact, we know that the bridge release will involve at least one >>>>> incompatible change. We will need to drop support for the --zookeeper >>>>> flags in the command-line tools. >>>>> >>>>> We've been preparing for this change for a long time. People have >>> spent >>>>> a lot of effort designing new APIs that can be used instead of the old >>>>> zookeeper-based code that some of the command-line tools used. We have >>>>> also deprecated the old ZK-based flags. But at the end of the day, it >>>>> is still an incompatible change. So it's unfortunately not possible >>> for >>>>> the >>>>> bridge release to be a 2.x release. >>>>> >>>>>> If that's not the case (i.e. if a single "bridge release" turns out >>> to >>>>>> be impractical), we should consider forking 3.0 while maintaining a >>>>>> line of Zookeeper-dependent Kafka in 2.x. That way 3.x can evolve >>>>>> dramatically without breaking the 2.x line. In particular, anything >>>>>> related to removing Zookeeper could land in pre-3.0 while every other >>>>>> feature targets 2.6. >>>>> >>>>> Just to be super clear about this, what we want to do here is support >>>>> operating in __either__ KIP-500 mode and legacy mode for a while. So >>> the >>>>> same branch will have support for both the old way and the new way of >>>>> managing metadata. >>>>> >>>>> This will allow us to get an "alpha" version of the KIP-500 mode out >>> early >>>>> for people to experiment with. It also greatly reduces the number of >>> Kafka >>>>> releases we have to make, and the amount of backporting we have to do. >>>>> >>>>>> >>>>>> If you are proposing 2.6 should be the "bridge release", I think this >>>>>> is premature given Kafka's time-based release schedule. If the bridge >>>>>> features happen to be merged before 2.6's feature freeze, then sure >>> -- >>>>>> let's make that the bridge release in retrospect. And if we get all >>>>>> the post-Zookeeper features merged before 2.7, I'm onboard with >>> naming >>>>>> it "3.0" instead. >>>>>> >>>>>> That said, we should aim to remove legacy MirrorMaker before 3.0 as >>>>>> well. I'm happy to drive that additional breaking change. Maybe 2.6 >>>>>> can be the "bridge" for MM2 as well. >>>>> >>>>> I don't have a strong opinion either way about this, but if we want to >>>>> remove the original MirrorMaker, we have to deprecate it first, >>> right? Are >>>>> we ready to do that? >>>>> >>>>> best, >>>>> Colin >>>>> >>>>>> >>>>>> Ryanne >>>>>> >>>>>> On Mon, May 4, 2020, 5:05 PM Colin McCabe <cmcc...@apache.org> >>> wrote: >>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> We've had a few proposals recently for incompatible changes. One >>> of >>>>>>> them is my KIP-604: Remove ZooKeeper Flags from the Administrative >>>>>>> Tools. The other is Boyang's KIP-590: Redirect ZK Mutation >>>>>>> Protocols to the Controller. I think it's time to start thinking >>>>>>> about Kafka 3.0. Specifically, I think we should move to 3.0 after >>>>>>> the 2.6 release. >>>>>>> >>>>>>> From the perspective of KIP-500, in Kafka 3.x we'd like to make >>>>>>> running in a ZooKeeper-less mode possible (but not yet the >>> default.) >>>>>>> This is the motivation behind KIP-590 and KIP-604, as well as some >>>>>>> of the other KIPs we've done recently. Since it will take some >>> time >>>>>>> to stabilize the new ZooKeeper-free Kafka code, we will hide it >>>>>>> behind an option initially. (We'll have a KIP describing this all >>> in >>>>>>> detail soon.) >>>>>>> >>>>>>> What does everyone think about having Kafka 3.0 come up next after >>>>>>> 2.6? Are there any other things we should change in the 2.6 -> 3.0 >>>>>>> transition? >>>>>>> >>>>>>> best, Colin >>>>>>> >>>>>> >>>>> >>>> >>> >> >> >> -- >> -- Guozhang >>
signature.asc
Description: OpenPGP digital signature
