Thanks Xintong and Jark for kicking off the great discussion! I checked the list carefully. The plans are detailed and most of the problems are covered Some of the ideas Chesnay mentioned, I think we should iterate in small steps and collect feedback in time Looking forward to the start of the work of Flink2.0, I am willing to provide assistance ~
Xintong Song <tonysong...@gmail.com> 于2023年4月25日周二 19:10写道： > > Hi everyone, > > I'd like to start a discussion on planning for a Flink 2.0 release. > > AFAIK, in the past years this topic has been mentioned from time to time, > in mailing lists, jira tickets and offline discussions. However, few > concrete steps have been taken, due to the significant determination and > efforts it requires and distractions from other prioritized focuses. After > a series of offline discussions in the recent weeks, with folks mostly from > our team internally as well as a few from outside Alibaba / Ververica > (thanks for insights from Becket and Robert), we believe it's time to kick > this off in the community. > > Below are some of our thoughts about the 2.0 release. Looking forward to > your opinions and feedback. > > > ## Why plan for release 2.0? > > > Flink 1.0.0 was released in March 2016. In the past 7 years, many new > features have been added and the project has become different from what it > used to be. So what is Flink now? What will it become in the next 3-5 > years? What do we think of Flink's position in the industry? We believe > it's time to rethink these questions, and draw a roadmap towards another > milestone, a milestone that worths a new major release. > > > In addition, we are still providing backwards compatibility (maybe not > perfectly but largely) with APIs that we designed and claimed stable 7 > years ago. While such backwards compatibility helps users to stick with the > latest Flink releases more easily, it sometimes, and more and more over > time, also becomes a burden for maintenance and a limitation for new > features and improvements. It's probably time to have a comprehensive > review and clean-up over all the public APIs. > > > Furthermore, next year is the 10th year for Flink as an Apache project. > Flink joined the Apache incubator in April 2014, and became a top-level > project in December 2014. That makes 2024 a perfect time for bringing out > the release 2.0 milestone. And for such a major release, we'd expect it > takes one year or even longer to prepare for, which means we probably > should start now. > > > ## What should we focus on in release 2.0? > > > - Roadmap discussion - How do we define and position Flink for now and > in future? This is probably something we lacked. I believe some people have > thought about it, but at least it's not explicitly discussed and aligned in > the community. Ideally, the 2.0 release should be a result of the roadmap. > - Breaking changes - Important improvements, bugfixes, technical debts > that involve breaking of API backwards compatibility, which can only be > carried out in major releases. > - With breaking API changes, we may need multiple 2.0-alpha/beta > versions to collect feedback. > - Key features - Significant features and improvements (e.g., new user > stories, architectural upgrades) that may change how users use Flink and > its position in the industry. Some items from this category may also > involve API breaking changes or significant behavior changes. > - There are also opinions that we should stay focused as much as > possible on the breaking changes only. Incremental / non-breaking > improvements and features, or anything that can be added in 2.x minor > releases, should not block the 2.0 release. > > It might be better to discuss the detailed technical items later in another > thread, to keep the current discussion focused on the overall proposal, and > to leave time for all parties to think about their technical plans. For > your reference, I've attached a preliminary list of work items proposed by > Alibaba / Ververica [1]. Note that the listed items are still being > carefully evaluated and prioritized, and may change in future. > > > ## How do we manage the release? > > > #### Release Process > > > We'd expect the release process for Flink 2.0 to be different from the 1.x > releases. > > > A major difference is that, we think the timeline-based release management > may not be suitable. The idea behind the timeline-based approach is that we > can have more frequent releases and deliver completed features to users > earlier, while incompleted features can be postponed to the next release > which won't be too late with the short release cycle. However, for breaking > changes that can only take place in major releases, the price for missing a > release is too high. > > > Alternatively, we probably should discuss and agree on a list of must-have > work items. That doesn't mean keep postponing the release upon a few > delayed features. In fact, we would need to closely monitor the progress of > the must-have items during the entire release cycle, making sure they are > taken care of by contributors with enough expertise and capacities. > > > #### Timeline > > > The release cycle should be decided according to the feature list, > especially the must-have items that we plan to do in the release. However, > a target feature freeze date would still be helpful when making the plan, > so that we don't pack too many things into the release. We propose to aim > for a feature freeze around mid 2024, so that in case must-have items are > delayed, we still have a good chance to make the release happen by the end > of 2024. > > > #### Branch > > > A longer release cycle also means we probably should keep shiping the 1.x > releases while preparing for the 2.0 release. We may cut a release-1 from > master, on which we can keep developing and release 1.x releases. The > version on the master branch will then become '2.0-SNAPSHOT'. > > > #### Release Manager > > > Given the new and to-be-explored release process, longer cycle and higher > synchronization requirements, we'd expect the 2.0 release to be more > challenging than previous 1.x releases. Therefore, we'd like to propose to > assemble a release management team with 4-5 experienced PMC members. Jark > and I would like to volunteer as 2 of the release managers. > > > Looking forward to your thoughts. > > > Best, > > Jark & Xintong > > > [1] > https://docs.google.com/document/d/1_PMGl5RuDQGlV99_gL3y7OiRsF0DgCk91Coua6hFXhE/edit?usp=sharing -- Best ConradJam
