Hi Everyone, +1 for the Release Managers. Thank you all for volunteering.
@Till Rohrmann <trohrm...@apache.org> A common sentiment that I have heard from many users is that upgrading off of 1.9 was very difficult. In particular, a lot of people struggled to understand the new memory model. Many users who required custom memory configurations in earlier versions assumed they should carry those configurations into latter versions and then found themselves with OOM and instability issues. The good news is Flink did what it was supposed to do and so for the majority dropping their custom configurations and just setting total process memory was the correct solution; this was not an issue of a buggy release. The problem is people do not read the release notes or fully understood the implications of the change. Back to Kurt's point, this transition seems to have left a bad taste in many mouths, slowing some user's adoption of newer versions. I don't know I have a solution to this problem. I think it is more communication than engineering, but I'm open to continuing the discussion. On Thu, Jun 3, 2021 at 5:04 AM Till Rohrmann <trohrm...@apache.org> wrote: > Thanks for volunteering as our release managers Xintong, Dawid and Joe! > > Thanks for starting the discussion about the release date Kurt. Personally, > I prefer in general shorter release cycles as it allows us to deliver > features faster and people feel less pressure to merge half-done features > last minute because they fear that they have to wait a long time if they > missed the train. Also, it forces us to make the release process less of a > stop-the-world event and cut down the costs of releases. > > On the other hand, if our users don't upgrade Flink fast enough, then > releasing more often won't have the effect of shipping features to our > users and getting feedback faster from our users faster. What I believe we > should try to do is to understand why upgrading Flink is so difficult for > them. What are the things preventing a quick upgrade and how can we improve > the situation for our users? Are our APIs not stable enough? Does Flink's > behavior changes too drastically between versions? Is the tooling for > upgrades lacking behind? Are they just cautious and don't want to use > bleeding edge software? > > If there is a problem that the majority of users is using an unsupported > version, then one solution could also be to extend the list of supported > Flink versions to the latest 3 versions, for example. > > About your 2) point I am a bit skeptical. I think that we will simply plan > more features and end up in the same situation wrt external contributions. > If it weren't the case, then it would also work with shorter release cycles > by simply planning less feature work and including the external > contribution, which could not be done in the past release, in the next > release. So in the end it is about what we plan for a release and not so > much how much time we have (assuming that we plan less if we have less time > and vice versa). > > Cheers, > Till > > On Thu, Jun 3, 2021 at 5:08 AM Kurt Young <ykt...@gmail.com> wrote: > > > Thanks for bringing this up. > > > > I have one thought about the release period. In a short word: shall we > try > > to extend the release period for 1 month? > > > > There are a couple of reasons why I want to bring up this proposal. > > > > 1) I observed that lots of users are actually far behind the current > Flink > > version. For example, we are now actively > > developing 1.14 but most users I know who have a migration or upgrade > plan > > are planning to upgrade to 1.12. This means > > we need to back port bug fixes to 1.12 and 1.13. If we extend the release > > period by 1 month, I think there may be some > > chances that users can have a proper time frame to upgrade to the > previous > > released version. Then we can have a > > good development cycle which looks like "actively developing the current > > version and making the previous version stable, > > not 2 ~ 3 versions before". Always far away from Flink's latest version > > also suppresses the motivation to contribute to Flink > > from users perspective. > > > > 2) Increasing the release period also eases the workload of committers > > which I think can improve the contributor experience. > > I have seen several times that when some contributors want to do some new > > features or improvements, we have to response > > with "sorry we are right now focusing with implementing/stabilizing > planned > > feature for this version", and the contributions are > > mostly like being stalled and never brought up again. > > > > BTW extending the release period also has downsides. It slows down the > > delivery speed of new features. And I'm also not > > sure how much it can improve the above 2 issues. > > > > Looking forward to hearing some feedback from the community, both users > and > > developers. > > > > Best, > > Kurt > > > > > > On Wed, Jun 2, 2021 at 8:39 PM JING ZHANG <beyond1...@gmail.com> wrote: > > > > > Hi Dawid, Joe & Xintong, > > > > > > Thanks for starting the discussion. > > > > > > I would like to polish Window TVFs[1][2] which is a popular feature in > > SQL > > > introduced in 1.13. > > > > > > The detailed items are as follows. > > > 1. Add more computations based on Window TVF > > > * Window Join (which is already merged in master branch) > > > * Window Table Function > > > * Window Deduplicate > > > 2. Finish related JIRA to improve user experience > > > * Add offset support for TUMBLE, HOP, session window > > > 3. Complement the missing functions compared to the group window, which > > is > > > a precondition of deprecating the legacy Grouped Window Function in the > > > later versions. > > > * Support Session windows > > > * Support allow-lateness > > > * Support retract input stream > > > * Support window TVF in batch mode > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-19604 > > > [2] > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-145%3A+Support+SQL+windowing+table-valued+function#FLIP145:SupportSQLwindowingtablevaluedfunction-CumulatingWindows > > > > > > Best regards, > > > JING ZHANG > > > > > > Xintong Song <xts...@apache.org> 于2021年6月2日周三 下午6:45写道： > > > > > > > Hi all, > > > > > > > > As 1.13 has been released for a while, I think it is a good time to > > start > > > > planning for the 1.14 release cycle. > > > > > > > > - Release managers: This time we'd like to have a team of 3 release > > > > managers. Dawid, Joe and I would like to volunteer for it. What do > you > > > > think about it? > > > > > > > > - Timeline: According to our approximate 4 months release period, we > > > > propose to aim for a feature freeze roughly in early August (which > > could > > > > mean something like early September for the 1.14. release). Does it > > work > > > > for everyone? > > > > > > > > - Collecting features: It would be helpful to have a rough overview > of > > > the > > > > new features that will likely be included in this release. We have > > > created > > > > a wiki page [1] for collecting such information. We'd like to kindly > > ask > > > > all committers to fill in the page with features that they intend to > > work > > > > on. > > > > > > > > We would also like to emphasize some aspects of the engineering > > process: > > > > > > > > - Stability of master: This has been an issue during the 1.13 feature > > > > freeze phase and it is still going on. We encourage every committer > to > > > not > > > > merge PRs through the Github button, but do this manually, with > caution > > > for > > > > the commits merged after the CI being triggered. It would be > > appreciated > > > to > > > > always build the project before merging to master. > > > > > > > > - Documentation: Please try to see documentation as an integrated > part > > of > > > > the engineering process and don't push it to the feature freeze phase > > or > > > > even after. You might even think about going documentation first. We, > > as > > > > the Flink community, are adding great stuff, that is pushing the > limits > > > of > > > > streaming data processors, with every release. We should also make > this > > > > stuff usable for our users by documenting it well. > > > > > > > > - Promotion of 1.14: What applies to documentation also applies to > all > > > the > > > > activity around the release. We encourage every contributor to also > > think > > > > about, plan and prepare activities like blog posts and talk, that > will > > > > promote and spread the release once it is done. > > > > > > > > Please let us know what you think. > > > > > > > > Thank you~ > > > > Dawid, Joe & Xintong > > > > > > > > [1] https://cwiki.apache.org/confluence/display/FLINK/1.14+Release > > > > > > > > > >
