Let’s discuss any and all ideas for improvement. As each is discussed we can figure out how to make them non-breaking, We all want Pulsar to improve.
We should encourage an open discussion where no idea is automatically bad or wrong. They can just be discussed without fear. Thanks, Dave > On Oct 10, 2022, at 3:05 PM, Joe F <joefranc...@gmail.com> wrote: > > I would prefer that we avoid using the term “breaking changes”, which is > too vague to convey any specific meaning. So let me try to bring some > clarity. > > > There have been many changes to implementations, APIs and data storage > formats in Pulsar (and book keeper also). I have deployed many of these > changes to production. And I know that Matteo and Rajan (and others too, > about whom I’m not up to date on) have implemented and deployed many such > changes. But none of those changes ever required taking the system > offline. NONE. > > > Pulsar was developed as a 24x7x365 system, and rolling upgrades and > rollbacks were a given. Like “this is water”, there was no special callout > needed for declaring this reality. No change, including enhancements to > wire protocols, broke client compatibility. Existing clients continued to > work; they may not be able to use all the new features. Use of new features > would require the app to be rebuilt anyway. (Checksums, e2e encryption are > examples) > > > We have even succeeded in getting Pulsar adopted for some use cases, just > because the complexity of upgrading from K’s old clients to new ones were > costly enough to allow consideration of an alternative like Pulsar. The > business cost of forcing a client upgrade can be significant, to the point > of this being unviable for business. That just cannot be hand-waved over > > > There have also been changes in storage formats(the ZK metadata change from > text to binary is an example). But through all such changes, compatibility > and upgradeability has been a given. There has never been a situation where > a live Pulsar upgrade was not possible, and a coordinated client upgrade > was mandatory. > > > So the question should not be about whether “signifcant” changes should > be made or not. Changes can be made and released in a way that breaks > *business*, or they can be made in a way that lets businesses sail > smoothly through that change. So the question is about how such changes > gets rolled out. > > > And to that question, my strong opinion is that any change that does not > allow a live/rolling upgrade or rollback, or anything that forces a client > to upgrade just to continue functioning, is a non-starter. All changes > can be made in a compatible, phased manner, and in a way that does not > penalise older versions ( older versions doing worse on new releases is > also not an acceptable way of making changes) Changes can be made in a > manner that make l A/B testing possible by the user, with limited risk, and > then choosing to a not go back. It has all been done in Pulsar before. > > > Would that be harder than just breaking stuff? Yes. But that is far more > preferable than forcing users to take a hit. > > > -joe > > On Sat, Oct 8, 2022 at 1:25 PM Rajan Dhabalia <rdhaba...@apache.org> wrote: > >> I would say first we should gather a list of changes which we want to >> target and find out which improvements really need major version release. >> We can take the Pulsar-1.0 to Pulsar-2.0 upgrade example to avoid major >> interruption and impact on existing systems and still achieve our goal. So, >> the first step is discovery of such features and then we can discuss how to >> introduce them in Pulsar with minimum impact on existing systems. >> >> Thanks, >> Rajan >> >> On Sat, Oct 8, 2022 at 1:05 PM Devin Bost <devin.b...@gmail.com> wrote: >> >>> I'm noticing some pushback on the idea of pre-emptively proposing any >> kind >>> of breaking upgrade that would necessitate cutting a 3.0 release. >>> I do understand the concern about introducing a breaking change... For a >>> distributed messaging application like Pulsar, if clients needed to be >>> simultaneously upgraded with brokers, that could be extremely difficult >> or >>> infeasible for companies to coordinate without treating it like a >> migration >>> to a new technology. >>> >>> At the same time, do we want to be completely closed to the possibility >>> that a breaking change could be required at some point in the future? If >> a >>> circumstance like that appears, those are the kinds of situations that >> can >>> lead to a fork. Are there certain kinds of breaking changes that are more >>> acceptable than others? >>> >>> Also, if the forward looking plan is to never introduce breaking changes, >>> when *would* we ever cut a Pulsar 3.x release? Do we have any criteria >> on >>> what kinds of changes would necessitate cutting a new major release but >>> would still be considered acceptable by the community? >>> >>> -- >>> Devin Bost >>> Sent from mobile >>> Cell: 801-400-4602 >>> >>> On Sat, Oct 8, 2022, 2:14 PM Rajan Dhabalia <rdhaba...@apache.org> >> wrote: >>> >>>> This sounds like the current state of Apache Pulsar has a lot of issues >>> and >>>> it requires fundamental design changes to make it promising which is >>>> definitely not true and I disagree with it. And I would be careful >>>> comparing with Kafka as I still don't think the Kafka release has >>> anything >>>> to do with Pulsar's improvement. I would still recommend to list down >> all >>>> the changes at one place so we can bring everyone on the same page. >>> discuss >>>> as a community and we make sure existing usecases continue using Pulsar >>> and >>>> not try to find Pulsar alternatives with incorrect disruption >> impression >>>> and efforts they might have to put to upgrade or maintain pulsar. >>>> >>>> Thanks, >>>> Rajan >>>> >>>> On Fri, Oct 7, 2022 at 7:49 PM Lari Hotari <lhot...@apache.org> wrote: >>>> >>>>> We could all have our own favorite names for this work. :) >>>>> >>>>> There's advice that you should disrupt yourself before someone >> disrupts >>>>> you. >>>>> Shouldn't we follow that advice for Apache Pulsar? We can disrupt >>> Pulsar >>>>> together with our Apache hats on. The catch is that since we are >> doing >>>>> this, we will be able to learn and improve Pulsar so that we stay >> ahead >>>> of >>>>> competition. Pulsar was long ways ahead of competition for so many >>> years, >>>>> but Kafka is finally catching up. Did Kafka surpass Pulsar in some >>>> aspects >>>>> with the recent 3.3 release, where Kraft became GA? That's a question >>>> that >>>>> many might be asking. Why wouldn't we rev up Pulsar's engine and show >>> the >>>>> tail lights to Kafka? >>>>> >>>>> We don't have to have deadlines or any restrictions like that right >>> now. >>>>> The sky's the limit. >>>>> Linus Torvalds has written a book called "Just for fun". I got my >> copy >>> of >>>>> this book signed by Linus himself in year 2000 at an event that the >>> book >>>>> publisher had organized in Finland. >>>>> >>>>> What if we did this "just for fun"? The intention could also be to >> beat >>>>> Kafka, but that could be a boring goal for many. What if we could >>> unleash >>>>> some talent that is among us and hasn't had a chance to show its full >>>>> potential? Opensource is about joy. It is about welcoming everyone to >>>> join. >>>>> Opensource should be egoless, although we must all admit that we >> don't >>>>> succeed in that aspect. We must fight our biases. >>>>> >>>>> Jarek Potiuk explains the importance of being welcoming for success >> at >>>>> Apache, in a 3-minute YouTube interview: >>>>> https://www.youtube.com/watch?v=Dx5kQnVFo7E >>>>> This interview is about Jarek's blog post "Success at Apache: >> Welcoming >>>>> communities strengthens the Apache way": >>>>> >>>>> >>>> >>> >> https://news.apache.org/foundation/entry/success-at-apache-welcoming-communities >>>>> I was pleased to meet Jarek at ApacheCon among so many other >> welcoming >>>>> personalities of the Apache community and the Apache Pulsar >> community. >>>>> >>>>> Goals have to be ambitious. What if we set the bar really high? >>>>> Apache Pulsar with 10 million topics in a cluster? >>>>> Why not go up to 100 million topics? >>>>> Just for fun. :) >>>>> >>>>> -Lari >>>>> >>>>> On 2022/10/07 22:53:59 Matteo Merli wrote: >>>>>> I actually disagree with the term "Pulsar Next Gen", because I >>> haven't >>>>>> seen any proposal for which that would make sense to me to be >> called >>>>>> so. >>>>>> >>>>>> Rajan: That's the whole point of breaking it down. If you >> accumulate >>>>>> many "big" changes it introduces a lot of risk for instabilities >> and >>>>>> incompatibilities. Breaking it down in multiple steps helps to see >>> the >>>>>> incremental changes and introduced them in a phased manner. >>>>>> >>>>>> >>>>>> -- >>>>>> Matteo Merli >>>>>> <matteo.me...@gmail.com> >>>>>> >>>>>> On Fri, Oct 7, 2022 at 3:37 PM Rajan Dhabalia < >> rdhaba...@apache.org> >>>>> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Can we get the list of changes at one place which we are planning >>> to >>>>> get as >>>>>>> part of 3.0. One thing I would like to see as a part of a major >>>>> release, it >>>>>>> CAN NOT impact existing usecases and users in any way which can >>> force >>>>> them >>>>>>> to upgrade the client library. Applications using < 3.0 version >>>> should >>>>>>> continue getting all the client and server side enhancements and >>> bug >>>>> fixes. >>>>>>> Failing to provide bug-fixes and features to client < 3.0 means >> we >>>> are >>>>>>> forcing them to upgrade client version by putting efforts to >> handle >>>> all >>>>>>> incompatibility. and that's something we should definitely >> prevent >>>>> because >>>>>>> Apache Pulsar is used by many large scale business usecases and >> we >>>>> should >>>>>>> accommodate and motivate them to continue using Apache Pulsar. >>>>>>> I understand as a Pulsar community we should always try to >> progress >>>> and >>>>>>> build better but not at the cost of losing or reducing the Apache >>>>> Pulsar >>>>>>> community. >>>>>>> >>>>>>> Thanks, >>>>>>> Rajan >>>>>>> >>>>>>> >>>>>>> On Fri, Oct 7, 2022 at 12:41 PM Lari Hotari <lhot...@apache.org> >>>>> wrote: >>>>>>> >>>>>>>> Thank you, Matteo. I agree that features should be delivered >>>>> continuously >>>>>>>> when that is possible. In this case, that might not apply. >>>>>>>> >>>>>>>> I also agree that calling this Pulsar 3.0 isn't necessarily >>> aligned >>>>> with >>>>>>>> PIP-175 since an LTS release is when the major version is >> bumped. >>>>> I'm fine >>>>>>>> in calling this "Pulsar Next Gen" or something that calls out >>> that >>>>> this is >>>>>>>> planning for making a major leap in Pulsar. >>>>>>>> >>>>>>>> There are several unresolved issues with PIP-45 and the Pulsar >>> Load >>>>>>>> balancer. The previously referred email threads contain a lot >> of >>>>> context to >>>>>>>> this. Resolving the issues efficiently will most likely result >> in >>>>> breaking >>>>>>>> changes, which will be the reason why it deserves a major >> version >>>>> upgrade. >>>>>>>> >>>>>>>> We have discussed it before that it's crucial to have a path to >>>>> migrate >>>>>>>> users when there are breaking changes. This should be covered >> in >>>> any >>>>> of the >>>>>>>> solutions that are introduced. Optimally, users of Pulsar would >>> be >>>>> able to >>>>>>>> upgrade seamlessly to Pulsar Next Gen / Pulsar 3.0, but rolling >>>> back >>>>> might >>>>>>>> not be directly supported. >>>>>>>> >>>>>>>> I am welcoming everyone to join this planning for the Apache >>> Pulsar >>>>> Next >>>>>>>> Gen architecture. Please check the first email in this thread >> for >>>>> details >>>>>>>> of context, and start participating and contributing today. The >>>> best >>>>> way to >>>>>>>> contribute is to participate in the email threads, since they >>>> contain >>>>>>>> details with better context. >>>>>>>> >>>>>>>> -Lari >>>>>>>> >>>>>>>> On 2022/10/07 18:03:00 Matteo Merli wrote: >>>>>>>>> Given the past experiences and the discussions that already >>>>> happened >>>>>>>>> around "PIP-175: Extend time based release process", the idea >>> is >>>> to >>>>>>>>> detach the 3.0 from "big-features" items or "incompatible >>>> changes". >>>>>>>>> >>>>>>>>> The changes are going to get included as they are ready, >> within >>>>>>>>> feature releases, and in a fully compatible way. We don't >> need >>> to >>>>>>>>> group them together and create unnecessary risk for the >> release >>>>>>>>> schedule and the users. >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Matteo Merli >>>>>>>>> <matteo.me...@gmail.com> >>>>>>>>> >>>>>>>>> On Fri, Oct 7, 2022 at 10:47 AM Lari Hotari < >>> lhot...@apache.org> >>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> Greeting from ApacheCon North America 2022 from New >> Orleans! >>>>>>>>>> We had a great conference with a dedicated Pulsar track. >>> Thanks >>>>> to all >>>>>>>> presenters and everyone who attended. The talks weren't >> recorded, >>>>> but the >>>>>>>> slides will be later on posted on the conference website [1]. >>>>>>>>>> >>>>>>>>>> At ApacheCon there were several presentations about "the >>> Apache >>>>> way" >>>>>>>> and what that means in practice. Based on that, we all know >> that >>> no >>>>> person >>>>>>>> is nominated as the CTO of Apache Pulsar who decides on Pulsar >>> 3.0 >>>>> and when >>>>>>>> that happens. It's us, the community, that serve that role >>>> together. >>>>> We >>>>>>>> come together as individuals with the Apache hat on. Everyone >> is >>>>> equal in >>>>>>>> the community, regardless of whether they are contributors, >>>>> committers or >>>>>>>> PMC members. >>>>>>>>>> We welcome everyone to participate. The small detail about >>>> voting >>>>>>>> shouldn't stop anyone from participating in any aspects of the >>>>> planning for >>>>>>>> the roadmap. >>>>>>>>>> >>>>>>>>>> I'll like to get the discussions going for Pulsar 3.0. We >>> don't >>>>> need a >>>>>>>> separate decision to start planning that. Please correct me if >>> I'm >>>>> wrong or >>>>>>>> if you have a different opinion. >>>>>>>>>> >>>>>>>>>> There are a few previous discussion threads that are >> related >>> to >>>>> Pulsar >>>>>>>> 3.0 planning. >>>>>>>>>> If you are interested in getting involved with Apache >> Pulsar >>>> 3.0 >>>>>>>> planning, I think that it makes sense for you to read these >>> threads >>>>>>>> carefully and reply to them. Please also suggest what you think >>>>> makes sense. >>>>>>>>>> >>>>>>>>>> PIP-45 related: >>>>>>>> >> https://lists.apache.org/thread/tvco1orf0hsyt59pjtfbwoq0vf6hfrcj >>>>>>>>>> Pulsar Load balancer / namespace bundle related: >>>>>>>>>> >>>> https://lists.apache.org/thread/roohoc9h2gthvmd7t81do4hfjs2gphpk >>>>>>>>>> renaming topics: >>>>>>>>>> >>>> https://lists.apache.org/thread/vrr75rrh4trqlp14objh3snlfvmzdrp2 >>>>>>>>>> backpressure: >>>>>>>> >> https://lists.apache.org/thread/v7xy57qfzbhopoqbm75s6ng8xlhbr2q6 >>>>>>>>>> >>>>>>>>>> Long list of Metadata inconsistency issues by Zac Bentley: >>>>>>>>>> https://github.com/apache/pulsar/issues/12555 >>>>>>>>>> That would be a good starting point to understanding the >> data >>>>>>>> inconsistency issues related to current PIP-45 design. Perhaps >>>> those >>>>> could >>>>>>>> be addressed already before Pulsar 3.0? >>>>>>>>>> >>>>>>>>>> I'm looking forward to everyone's participation in the >> Apache >>>>> Pulsar >>>>>>>> 3.0 planning discussions. >>>>>>>>>> >>>>>>>>>> Best Regards, >>>>>>>>>> >>>>>>>>>> -Lari >>>>>>>>>> >>>>>>>>>> 1 - https://www.apachecon.com/acna2022/schedule.html >>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >>> >>
