Hi, I agree with others in this thread that everyone should be focused on making the release happen and not push their own agenda. It is generally difficult to manage releases with so many interests and moving parts. Many feel the urgency to complete what they were originally hoping to complete. But let's face reality of software development, most things take longer than estimated and that should be no reason to delay a release that brings many other good things to the users. The train leaves the station, whether you are there or not. And the next train is certain to arrive :)
I think it would help to establish the process where the release manager is in the know of what gets merged to a branch and has the ability to stop merges that are not appropriate. Perhaps that can be documented more clearly but it is also good to be able to rely on the responsibility of committers to do the right thing. If completing the release in the interest of the community has priority, that should be almost automatic. Thanks, Thomas On Fri, Aug 9, 2019 at 11:04 AM Xuefu Z <usxu...@gmail.com> wrote: > Thanks all for sharing thoughts. I agree with Tzu-li that release schedule > deserves a separate discussion. For the PR in question, I'd concur with > Till that we keep it due to its limited scope of impact. However, it would > be okay to me revert it if it's found to pose a big risk to the stability > of the release. > > Thanks, > Xuefu > > On Fri, Aug 9, 2019 at 6:45 AM Till Rohrmann <trohrm...@apache.org> wrote: > > > I can only second Stephan's, Timo's, Piotr's and Gordon's points. > > > > I also want to re-emphasize the importance of sticking to the agreed > > processes for the community. If people merge features at free will after > > the feature freeze, then they willingly put the release and with that the > > work of the whole Flink community at risk. The idea of time based > releases > > is that we release regularly so that a feature which does not make it > until > > the feature freeze, will be released with the next (soon to come) > release. > > That way a single feature won't delay the delivery of plenty of other > > completed and helpful features. For reference, here is the wiki page > > explaining Flink's agreed upon release process [1]. > > > > For the problem at hand, I would actually vote for a pragmatic approach > > even though I'm aware that this can easily backfire. So far the merged > > feature does not seem to destabilize the Flink build. Moreover, it seems > to > > only affect the flink-table-planner-blink and the flink-connector-hive > > modules. Hence if something is broken, then it would should hopefully > > affect only these components. > > > > Glancing over the code, I think it contains some clear signs that the > code > > has been rushed. For example, I could find several comments "TODO hack > > code" in the committed code which tells me that we have committed instant > > technological debt. Normally, this would be an argument to revert the > code > > because not only does it have the potential to bite us in the ass later > on > > but it is usually also a sign of immaturity. However, since we consider > the > > Hive connector and also the Blink runner a somewhat preview feature, I > > would leave it up to Kurt to make the call whether he wants to live with > > this debt (and consequently having to deal with it in the releases to > > come). > > > > [1] > https://cwiki.apache.org/confluence/display/FLINK/Time-based+releases > > > > Cheers, > > Till > > > > On Fri, Aug 9, 2019 at 2:25 PM Tzu-Li (Gordon) Tai <tzuli...@apache.org> > > wrote: > > > > > Hi, > > > > > > Just as a reminder, I think we should not focus here on discussing > > whether > > > or not time-based releases is a suitable release model for us. > > > That doesn’t seem to be the original intent of this thread, and is > > > therefore more suited for a separate discussion with the community, > > > especially taking into account that the time-based release model was > > agreed > > > upon by the community in the past. > > > > > > Regarding the definition and importance of feature freeze, I simply > want > > to > > > re-emphasize what Stephan, Timo, and Piotr had already mentioned. > > > A big part of a feature freeze is to let the new features and changes > > > settle down and allow the community to establish meaningful testing > > efforts > > > over a period of time, and drive the feature freeze release branch > > towards > > > a releasable state. > > > > > > With that in mind, specifically for FLINK-13225, I’m personally in > favor > > of > > > reverting the changes in the release-1.9 branch. > > > We have always allowed for flexibility (after informed discussions with > > the > > > community) throughout the release testing phase, even in previous major > > > releases. > > > But in this case, since we are already way past feature freeze (June > > 28th) > > > and that the first official voting RC for release was already expected > to > > > be created very soon by the end of this week [1], IMO it is dangerous > to > > > include them now as it potentially invalidates the testing efforts so > far > > > from other community members. > > > > > > Cheers, > > > Gordon > > > > > > [1] > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Progress-updates-for-Apache-Flink-1-9-0-release-tp30565p31426.html > > > > > > On Fri, Aug 9, 2019 at 8:25 AM Piotr Nowojski <pi...@ververica.com> > > wrote: > > > > > > > Hi Xuefu, > > > > > > > > As others have mentioned, time based releases were decided because of > > > some > > > > good reasons (more predictability for users so that they can take > that > > > into > > > > account for planning next upgrades). That doesn’t mean we can not > allow > > > for > > > > some flexibility: we do, we have already postponed this feature > freeze > > by > > > > at least a couple of weeks. If we start postponing it more, that will > > > > overlap with next release. > > > > > > > > However the key point is that feature freeze is also there for very > > good > > > > reason: to make sure that release will be stable. Because of that, as > > > > Stephan wrote we can not merge features (especially big ones) to the > > > > release branch as we wish since some features: > > > > 1. just need some time exposure on the CI to discover intermittent > > issues > > > > 2. need to have specific pre releases tests planned > > > > 3. if we have already done some pre releases tests on a RC candidate, > > If > > > > we merge new feature after that, those tests might be invalid > > > > > > > > I fully agree with Stephan (as it was already discussed in "Merging > PR > > to > > > > release branches during RC voting” mailing list thread [1]), that > > merging > > > > things to the release branch should be strictly coordinated with the > > > > release manager. This doesn’t mean that your feature will not make it > > to > > > > the given release, but just inform the release manager, discuss it > and > > > make > > > > an informed decision. Even for this release, there were examples were > > we > > > > merged something to release branch after the feature freeze after > > > > discussing it with the release manager. > > > > > > > > Piotrek > > > > > > > > [1] > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Merging-PR-to-release-branches-during-RC-voting-td22461.html > > > > < > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Merging-PR-to-release-branches-during-RC-voting-td22461.html > > > > > > > > > > > > > > > > > On 8 Aug 2019, at 20:10, Xuefu Z <usxu...@gmail.com> wrote: > > > > > > > > > > Hi Timo, > > > > > > > > > > Thanks for sharing your opinion. By wastefulness, I meant we had > > > planned > > > > > and done much work ending up with not being useful in the released > > > > product. > > > > > Instead of making many partial features, we'd rather make fewer but > > > > > complete features. We expected a good integration with Hive in 1.9, > > but > > > > for > > > > > one reason or another, the scope was reduced over and over again. > > This > > > is > > > > > rather frustrating. > > > > > > > > > > As to whether the PR was planned, I'd say yes because it's what > Hive > > > > > Integration needed and Hive integration plan clearly had the > feature > > of > > > > > supporting Hive UDF. > > > > > > > > > > Hi Stephan, > > > > > > > > > > Thanks for your opinions as well. As to time-based release, if > it's a > > > > > general guideline I think it's fine. However, I think certain > > planning > > > > and > > > > > flexibility wouldn't hurt. Theoretically, we could have nothing new > > or > > > > > complete to release when the scheduled timer hits. In this case, it > > > > > probably makes sense to delay the release until a meaningful > release > > is > > > > > there. We cannot so de-emphasize the value of the planning as well > > > > because > > > > > that's where we set up goals of solving certain user problems or > > > meeting > > > > > some business goals. If we are too far off the original > expectation, > > it > > > > > might make sense to have some flexibility on timing. > > > > > > > > > > Anyway, all I have said was about how we balance between timely > > > releasing > > > > > and maintaining some criteria on what would be released, not about > > > > > downplaying the virtue of time-based releasing. > > > > > > > > > > Thanks, > > > > > Xuefu > > > > > > > > > > On Thu, Aug 8, 2019 at 10:18 AM Stephan Ewen <se...@apache.org> > > wrote: > > > > > > > > > >> Again, feature freeze is not about "what was planned", it is a > about > > > > what > > > > >> is ready. Otherwise, it is completely unplannable when a release > > would > > > > >> come. > > > > >> Everyone has a pet feature they want to see in. If everyone just > > makes > > > > >> decisions by themselves and pushes, we can never get anywhere. > > > > >> > > > > >> Disagreement with the current practice can be raised and > discussed. > > > > >> But one sided decisions to just ignore the established rules > because > > > of > > > > >> disagreement is not acceptable. That's not how this (or any) open > > > source > > > > >> community can work. > > > > >> > > > > >> If we want to keep building an open and fair community and still > > > release > > > > >> high quality software, we have to be very clear about this. > > > > >> > > > > >> > > > > >> > > > > >> On Thu, Aug 8, 2019 at 5:29 PM Timo Walther <twal...@apache.org> > > > wrote: > > > > >> > > > > >>> Hi Xuefu, > > > > >>> > > > > >>> I disagree with "all those work would be wasted/useless", it > would > > > just > > > > >>> take effect 3 months later. > > > > >>> > > > > >>> Regarding "I don't see eye to eye on how and when we had decided > a > > > > >>> feature freeze", there was an official [ANNOUNCE] email that > > targeted > > > > >>> June 28 [1]. I think nobody is super strict about such a date and > > an > > > > >>> additional day or two but we need to stop merging into a branch > to > > > > >>> ensure build stability. > > > > >>> > > > > >>> The Flink community decided on time-based releases some time ago, > > of > > > > >>> course we could discuss this policy again. But generally speaking > > we > > > > >>> should release quickly because every release contains nice > features > > > > that > > > > >>> users are waiting for. The PR in question was not listed in the > > > intial > > > > >>> feature discussion [2] and mentioned for the first time mid/end > of > > > > June. > > > > >>> > > > > >>> Personally, for the next release, I would prefer to vote on a > list > > of > > > > >>> FLIP topics that qualify for a release (given that they are > > finished > > > in > > > > >>> time with the expected quality). > > > > >>> > > > > >>> What do you think? > > > > >>> > > > > >>> Thanks, > > > > >>> Timo > > > > >>> > > > > >>> [1] > > > > >>> > > > > >>> > > > > >> > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Feature-freeze-for-Apache-Flink-1-9-0-release-tp29751.html > > > > >>> [2] > > > > >>> > > > > >>> > > > > >> > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Features-for-Apache-Flink-1-9-0-td28701.html > > > > >>> > > > > >>> Am 08.08.19 um 15:43 schrieb Xuefu Z: > > > > >>>> Hi all, > > > > >>>> > > > > >>>> I understand the merged PR is a feature, but it's something we > had > > > > >>> planned > > > > >>>> and requested for a long time. In fact, at Hive connector side, > > we > > > > >> have > > > > >>>> done a lot of work (supporting hive udf). Without this PR, all > > those > > > > >> work > > > > >>>> would be wasted and Hive feature itself in 1.9 would also be > close > > > to > > > > >>> being > > > > >>>> useless. > > > > >>>> > > > > >>>> I also agree that feature freeze means something and has its > > > > >> importance. > > > > >>> On > > > > >>>> the other hand, I don't see eye to eye on how and when we had > > > decided > > > > a > > > > >>>> feature freeze should be in place. To me, our feature freeze > > seems > > > to > > > > >> be > > > > >>>> time based. That is, we determine a time by which feature freeze > > > will > > > > >>>> happen, irregardless original feature plan. As a result, this > > > practice > > > > >>>> incurs a great deal of randomness, leaving many planned feature > > half > > > > >>> baked. > > > > >>>> The question is really about how we balance releasing something > > in > > > > >> time > > > > >>>> vs releasing something usable. This might be a great chance for > > us > > > to > > > > >>>> meditate on this topic. > > > > >>>> > > > > >>>> The PR in question is requested by me, and its importance to > Hive > > > > >>> connector > > > > >>>> makes me stand by my request. On the other hand, if the PR has > > > > anything > > > > >>> to > > > > >>>> improve, I'm all for it. > > > > >>>> > > > > >>>> Thanks, > > > > >>>> Xuefu > > > > >>>> > > > > >>>> On Thu, Aug 8, 2019 at 2:59 AM Timo Walther <twal...@apache.org > > > > > > >> wrote: > > > > >>>> > > > > >>>>> Hi Kurt, > > > > >>>>> > > > > >>>>> I posted my opinion around this particular example in > > FLINK-13225. > > > > >>>>> > > > > >>>>> Regarding the definition of "feature freeze": I think it is > good > > to > > > > >>>>> write down more of the implicit processes that we had in the > > past. > > > > The > > > > >>>>> bylaws, coding guidelines, and a better FLIP process are very > > good > > > > >> steps > > > > >>>>> towards the right direction. However, not everything can be > > written > > > > >> down > > > > >>>>> and formulized. We should also remind ourselves of basic > software > > > > >>>>> engineering principles. Merging a feature shortly before the > > actual > > > > >>>>> release is always dangerous. A feature needs time to settle > down > > > and > > > > >> be > > > > >>>>> tested for side-effects etc. Merging a feature with a lot of > > > > spaghetti > > > > >>>>> code, reflection magic, and a single IT case is not a complete > > > > feature > > > > >>>>> that is worth merging. > > > > >>>>> > > > > >>>>> I hope we can improve here for the next release. Thanks for the > > > open > > > > >>>>> discussion. > > > > >>>>> > > > > >>>>> Regards, > > > > >>>>> Timo > > > > >>>>> > > > > >>>>> > > > > >>>>> Am 08.08.19 um 11:11 schrieb Kurt Young: > > > > >>>>>> Hi Stephan, > > > > >>>>>> > > > > >>>>>> Thanks for bringing this up. I think it's important and a good > > > time > > > > >> to > > > > >>>>>> discuss what > > > > >>>>>> does *feature freeze* really means. At least to me, seems I > have > > > > some > > > > >>>>>> misunderstandings with this comparing to other community > > members. > > > > But > > > > >>> as > > > > >>>>>> you > > > > >>>>>> pointed out in the jira and also in this mail, I think your > > > > >>> understanding > > > > >>>>>> makes sense > > > > >>>>>> to me. > > > > >>>>>> > > > > >>>>>> Maybe we can have a conclusion in the thread and put this into > > the > > > > >>>>> project > > > > >>>>>> bylaws > > > > >>>>>> which are under discussion? > > > > >>>>>> > > > > >>>>>> Regarding to FLINK-13225, I would like to hear other's opinion > > > since > > > > >> I > > > > >>>>>> merged it. But > > > > >>>>>> I would like to revert it if someone voted for reverting it. > > > > >>>>>> > > > > >>>>>> Sorry for the inconvenience I caused. > > > > >>>>>> > > > > >>>>>> Best, > > > > >>>>>> Kurt > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> On Thu, Aug 8, 2019 at 4:46 PM Stephan Ewen <se...@apache.org > > > > > > >> wrote: > > > > >>>>>> > > > > >>>>>>> Hi all! > > > > >>>>>>> > > > > >>>>>>> I would like to bring this topic up, because we saw quite a > few > > > > >>> "secret" > > > > >>>>>>> post-feature-freeze feature merges. > > > > >>>>>>> The latest example was > > > > >>>>> https://issues.apache.org/jira/browse/FLINK-13225 > > > > >>>>>>> I would like to make sure that we are all on the same page on > > > what > > > > a > > > > >>>>>>> feature freeze means and how to handle possible additions > after > > > the > > > > >>>>> feature > > > > >>>>>>> freeze. > > > > >>>>>>> My understanding was the following, and I assume that this > was > > > also > > > > >>> the > > > > >>>>>>> understanding of the community when we started establishing > the > > > > >>> release > > > > >>>>>>> practice: > > > > >>>>>>> > > > > >>>>>>> - Feature freeze is the date until new features can be > > merged. > > > > >>>>>>> - After the feature freeze, we only merge bug fixes, > release > > > > >>> relevant > > > > >>>>>>> tests (end to end tests), and documentation. > > > > >>>>>>> - Features should already be stable and have tests. It is > > not > > > > >>> okay to > > > > >>>>>>> "get a foot in the door" before feature freeze by merging > > > something > > > > >>>>> that is > > > > >>>>>>> not ready (as a placeholder) and then fixing it post feature > > > > freeze. > > > > >>>>>>> - Extending functionality to new components is not a bug > > fix, > > > it > > > > >>> is a > > > > >>>>>>> feature. > > > > >>>>>>> - If someone wants to add a minor feature after the > feature > > > > >>> freeze, > > > > >>>>> and > > > > >>>>>>> there is a good reason for that, it should be explicitly > > > discussed. > > > > >> If > > > > >>>>>>> there is no objection, it can be merged. > > > > >>>>>>> > > > > >>>>>>> Please let me know if you have a different understanding of > > what > > > > >>> feature > > > > >>>>>>> freeze means. > > > > >>>>>>> > > > > >>>>>>> Regarding the issue of FLINK-13225 > > > > >>>>>>> <https://issues.apache.org/jira/browse/FLINK-13225>? > > > > >>>>>>> - Should we keep it? > > > > >>>>>>> - Should we revert it in the release-1.9 branch and only > > keep > > > it > > > > >>> for > > > > >>>>>>> master? > > > > >>>>>>> > > > > >>>>>>> Best, > > > > >>>>>>> Stephan > > > > >>>>>>> > > > > >>>>> > > > > >>> > > > > >>> > > > > >> > > > > > > > > > > > > > > > -- > > > > > Xuefu Zhang > > > > > > > > > > "In Honey We Trust!" > > > > > > > > > > > > > > > > -- > Xuefu Zhang > > "In Honey We Trust!" >
