Thanks all for sharing thoughts. I agree with Tzu-li that release schedule deserves a separate discussion. For the PR in question, I'd concur with Till that we keep it due to its limited scope of impact. However, it would be okay to me revert it if it's found to pose a big risk to the stability of the release.
Thanks, Xuefu On Fri, Aug 9, 2019 at 6:45 AM Till Rohrmann <trohrm...@apache.org> wrote: > I can only second Stephan's, Timo's, Piotr's and Gordon's points. > > I also want to re-emphasize the importance of sticking to the agreed > processes for the community. If people merge features at free will after > the feature freeze, then they willingly put the release and with that the > work of the whole Flink community at risk. The idea of time based releases > is that we release regularly so that a feature which does not make it until > the feature freeze, will be released with the next (soon to come) release. > That way a single feature won't delay the delivery of plenty of other > completed and helpful features. For reference, here is the wiki page > explaining Flink's agreed upon release process [1]. > > For the problem at hand, I would actually vote for a pragmatic approach > even though I'm aware that this can easily backfire. So far the merged > feature does not seem to destabilize the Flink build. Moreover, it seems to > only affect the flink-table-planner-blink and the flink-connector-hive > modules. Hence if something is broken, then it would should hopefully > affect only these components. > > Glancing over the code, I think it contains some clear signs that the code > has been rushed. For example, I could find several comments "TODO hack > code" in the committed code which tells me that we have committed instant > technological debt. Normally, this would be an argument to revert the code > because not only does it have the potential to bite us in the ass later on > but it is usually also a sign of immaturity. However, since we consider the > Hive connector and also the Blink runner a somewhat preview feature, I > would leave it up to Kurt to make the call whether he wants to live with > this debt (and consequently having to deal with it in the releases to > come). > > [1] https://cwiki.apache.org/confluence/display/FLINK/Time-based+releases > > Cheers, > Till > > On Fri, Aug 9, 2019 at 2:25 PM Tzu-Li (Gordon) Tai <tzuli...@apache.org> > wrote: > > > Hi, > > > > Just as a reminder, I think we should not focus here on discussing > whether > > or not time-based releases is a suitable release model for us. > > That doesn’t seem to be the original intent of this thread, and is > > therefore more suited for a separate discussion with the community, > > especially taking into account that the time-based release model was > agreed > > upon by the community in the past. > > > > Regarding the definition and importance of feature freeze, I simply want > to > > re-emphasize what Stephan, Timo, and Piotr had already mentioned. > > A big part of a feature freeze is to let the new features and changes > > settle down and allow the community to establish meaningful testing > efforts > > over a period of time, and drive the feature freeze release branch > towards > > a releasable state. > > > > With that in mind, specifically for FLINK-13225, I’m personally in favor > of > > reverting the changes in the release-1.9 branch. > > We have always allowed for flexibility (after informed discussions with > the > > community) throughout the release testing phase, even in previous major > > releases. > > But in this case, since we are already way past feature freeze (June > 28th) > > and that the first official voting RC for release was already expected to > > be created very soon by the end of this week [1], IMO it is dangerous to > > include them now as it potentially invalidates the testing efforts so far > > from other community members. > > > > Cheers, > > Gordon > > > > [1] > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Progress-updates-for-Apache-Flink-1-9-0-release-tp30565p31426.html > > > > On Fri, Aug 9, 2019 at 8:25 AM Piotr Nowojski <pi...@ververica.com> > wrote: > > > > > Hi Xuefu, > > > > > > As others have mentioned, time based releases were decided because of > > some > > > good reasons (more predictability for users so that they can take that > > into > > > account for planning next upgrades). That doesn’t mean we can not allow > > for > > > some flexibility: we do, we have already postponed this feature freeze > by > > > at least a couple of weeks. If we start postponing it more, that will > > > overlap with next release. > > > > > > However the key point is that feature freeze is also there for very > good > > > reason: to make sure that release will be stable. Because of that, as > > > Stephan wrote we can not merge features (especially big ones) to the > > > release branch as we wish since some features: > > > 1. just need some time exposure on the CI to discover intermittent > issues > > > 2. need to have specific pre releases tests planned > > > 3. if we have already done some pre releases tests on a RC candidate, > If > > > we merge new feature after that, those tests might be invalid > > > > > > I fully agree with Stephan (as it was already discussed in "Merging PR > to > > > release branches during RC voting” mailing list thread [1]), that > merging > > > things to the release branch should be strictly coordinated with the > > > release manager. This doesn’t mean that your feature will not make it > to > > > the given release, but just inform the release manager, discuss it and > > make > > > an informed decision. Even for this release, there were examples were > we > > > merged something to release branch after the feature freeze after > > > discussing it with the release manager. > > > > > > Piotrek > > > > > > [1] > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Merging-PR-to-release-branches-during-RC-voting-td22461.html > > > < > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Merging-PR-to-release-branches-during-RC-voting-td22461.html > > > > > > > > > > > > > On 8 Aug 2019, at 20:10, Xuefu Z <usxu...@gmail.com> wrote: > > > > > > > > Hi Timo, > > > > > > > > Thanks for sharing your opinion. By wastefulness, I meant we had > > planned > > > > and done much work ending up with not being useful in the released > > > product. > > > > Instead of making many partial features, we'd rather make fewer but > > > > complete features. We expected a good integration with Hive in 1.9, > but > > > for > > > > one reason or another, the scope was reduced over and over again. > This > > is > > > > rather frustrating. > > > > > > > > As to whether the PR was planned, I'd say yes because it's what Hive > > > > Integration needed and Hive integration plan clearly had the feature > of > > > > supporting Hive UDF. > > > > > > > > Hi Stephan, > > > > > > > > Thanks for your opinions as well. As to time-based release, if it's a > > > > general guideline I think it's fine. However, I think certain > planning > > > and > > > > flexibility wouldn't hurt. Theoretically, we could have nothing new > or > > > > complete to release when the scheduled timer hits. In this case, it > > > > probably makes sense to delay the release until a meaningful release > is > > > > there. We cannot so de-emphasize the value of the planning as well > > > because > > > > that's where we set up goals of solving certain user problems or > > meeting > > > > some business goals. If we are too far off the original expectation, > it > > > > might make sense to have some flexibility on timing. > > > > > > > > Anyway, all I have said was about how we balance between timely > > releasing > > > > and maintaining some criteria on what would be released, not about > > > > downplaying the virtue of time-based releasing. > > > > > > > > Thanks, > > > > Xuefu > > > > > > > > On Thu, Aug 8, 2019 at 10:18 AM Stephan Ewen <se...@apache.org> > wrote: > > > > > > > >> Again, feature freeze is not about "what was planned", it is a about > > > what > > > >> is ready. Otherwise, it is completely unplannable when a release > would > > > >> come. > > > >> Everyone has a pet feature they want to see in. If everyone just > makes > > > >> decisions by themselves and pushes, we can never get anywhere. > > > >> > > > >> Disagreement with the current practice can be raised and discussed. > > > >> But one sided decisions to just ignore the established rules because > > of > > > >> disagreement is not acceptable. That's not how this (or any) open > > source > > > >> community can work. > > > >> > > > >> If we want to keep building an open and fair community and still > > release > > > >> high quality software, we have to be very clear about this. > > > >> > > > >> > > > >> > > > >> On Thu, Aug 8, 2019 at 5:29 PM Timo Walther <twal...@apache.org> > > wrote: > > > >> > > > >>> Hi Xuefu, > > > >>> > > > >>> I disagree with "all those work would be wasted/useless", it would > > just > > > >>> take effect 3 months later. > > > >>> > > > >>> Regarding "I don't see eye to eye on how and when we had decided a > > > >>> feature freeze", there was an official [ANNOUNCE] email that > targeted > > > >>> June 28 [1]. I think nobody is super strict about such a date and > an > > > >>> additional day or two but we need to stop merging into a branch to > > > >>> ensure build stability. > > > >>> > > > >>> The Flink community decided on time-based releases some time ago, > of > > > >>> course we could discuss this policy again. But generally speaking > we > > > >>> should release quickly because every release contains nice features > > > that > > > >>> users are waiting for. The PR in question was not listed in the > > intial > > > >>> feature discussion [2] and mentioned for the first time mid/end of > > > June. > > > >>> > > > >>> Personally, for the next release, I would prefer to vote on a list > of > > > >>> FLIP topics that qualify for a release (given that they are > finished > > in > > > >>> time with the expected quality). > > > >>> > > > >>> What do you think? > > > >>> > > > >>> Thanks, > > > >>> Timo > > > >>> > > > >>> [1] > > > >>> > > > >>> > > > >> > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Feature-freeze-for-Apache-Flink-1-9-0-release-tp29751.html > > > >>> [2] > > > >>> > > > >>> > > > >> > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Features-for-Apache-Flink-1-9-0-td28701.html > > > >>> > > > >>> Am 08.08.19 um 15:43 schrieb Xuefu Z: > > > >>>> Hi all, > > > >>>> > > > >>>> I understand the merged PR is a feature, but it's something we had > > > >>> planned > > > >>>> and requested for a long time. In fact, at Hive connector side, > we > > > >> have > > > >>>> done a lot of work (supporting hive udf). Without this PR, all > those > > > >> work > > > >>>> would be wasted and Hive feature itself in 1.9 would also be close > > to > > > >>> being > > > >>>> useless. > > > >>>> > > > >>>> I also agree that feature freeze means something and has its > > > >> importance. > > > >>> On > > > >>>> the other hand, I don't see eye to eye on how and when we had > > decided > > > a > > > >>>> feature freeze should be in place. To me, our feature freeze > seems > > to > > > >> be > > > >>>> time based. That is, we determine a time by which feature freeze > > will > > > >>>> happen, irregardless original feature plan. As a result, this > > practice > > > >>>> incurs a great deal of randomness, leaving many planned feature > half > > > >>> baked. > > > >>>> The question is really about how we balance releasing something > in > > > >> time > > > >>>> vs releasing something usable. This might be a great chance for > us > > to > > > >>>> meditate on this topic. > > > >>>> > > > >>>> The PR in question is requested by me, and its importance to Hive > > > >>> connector > > > >>>> makes me stand by my request. On the other hand, if the PR has > > > anything > > > >>> to > > > >>>> improve, I'm all for it. > > > >>>> > > > >>>> Thanks, > > > >>>> Xuefu > > > >>>> > > > >>>> On Thu, Aug 8, 2019 at 2:59 AM Timo Walther <twal...@apache.org> > > > >> wrote: > > > >>>> > > > >>>>> Hi Kurt, > > > >>>>> > > > >>>>> I posted my opinion around this particular example in > FLINK-13225. > > > >>>>> > > > >>>>> Regarding the definition of "feature freeze": I think it is good > to > > > >>>>> write down more of the implicit processes that we had in the > past. > > > The > > > >>>>> bylaws, coding guidelines, and a better FLIP process are very > good > > > >> steps > > > >>>>> towards the right direction. However, not everything can be > written > > > >> down > > > >>>>> and formulized. We should also remind ourselves of basic software > > > >>>>> engineering principles. Merging a feature shortly before the > actual > > > >>>>> release is always dangerous. A feature needs time to settle down > > and > > > >> be > > > >>>>> tested for side-effects etc. Merging a feature with a lot of > > > spaghetti > > > >>>>> code, reflection magic, and a single IT case is not a complete > > > feature > > > >>>>> that is worth merging. > > > >>>>> > > > >>>>> I hope we can improve here for the next release. Thanks for the > > open > > > >>>>> discussion. > > > >>>>> > > > >>>>> Regards, > > > >>>>> Timo > > > >>>>> > > > >>>>> > > > >>>>> Am 08.08.19 um 11:11 schrieb Kurt Young: > > > >>>>>> Hi Stephan, > > > >>>>>> > > > >>>>>> Thanks for bringing this up. I think it's important and a good > > time > > > >> to > > > >>>>>> discuss what > > > >>>>>> does *feature freeze* really means. At least to me, seems I have > > > some > > > >>>>>> misunderstandings with this comparing to other community > members. > > > But > > > >>> as > > > >>>>>> you > > > >>>>>> pointed out in the jira and also in this mail, I think your > > > >>> understanding > > > >>>>>> makes sense > > > >>>>>> to me. > > > >>>>>> > > > >>>>>> Maybe we can have a conclusion in the thread and put this into > the > > > >>>>> project > > > >>>>>> bylaws > > > >>>>>> which are under discussion? > > > >>>>>> > > > >>>>>> Regarding to FLINK-13225, I would like to hear other's opinion > > since > > > >> I > > > >>>>>> merged it. But > > > >>>>>> I would like to revert it if someone voted for reverting it. > > > >>>>>> > > > >>>>>> Sorry for the inconvenience I caused. > > > >>>>>> > > > >>>>>> Best, > > > >>>>>> Kurt > > > >>>>>> > > > >>>>>> > > > >>>>>> On Thu, Aug 8, 2019 at 4:46 PM Stephan Ewen <se...@apache.org> > > > >> wrote: > > > >>>>>> > > > >>>>>>> Hi all! > > > >>>>>>> > > > >>>>>>> I would like to bring this topic up, because we saw quite a few > > > >>> "secret" > > > >>>>>>> post-feature-freeze feature merges. > > > >>>>>>> The latest example was > > > >>>>> https://issues.apache.org/jira/browse/FLINK-13225 > > > >>>>>>> I would like to make sure that we are all on the same page on > > what > > > a > > > >>>>>>> feature freeze means and how to handle possible additions after > > the > > > >>>>> feature > > > >>>>>>> freeze. > > > >>>>>>> My understanding was the following, and I assume that this was > > also > > > >>> the > > > >>>>>>> understanding of the community when we started establishing the > > > >>> release > > > >>>>>>> practice: > > > >>>>>>> > > > >>>>>>> - Feature freeze is the date until new features can be > merged. > > > >>>>>>> - After the feature freeze, we only merge bug fixes, release > > > >>> relevant > > > >>>>>>> tests (end to end tests), and documentation. > > > >>>>>>> - Features should already be stable and have tests. It is > not > > > >>> okay to > > > >>>>>>> "get a foot in the door" before feature freeze by merging > > something > > > >>>>> that is > > > >>>>>>> not ready (as a placeholder) and then fixing it post feature > > > freeze. > > > >>>>>>> - Extending functionality to new components is not a bug > fix, > > it > > > >>> is a > > > >>>>>>> feature. > > > >>>>>>> - If someone wants to add a minor feature after the feature > > > >>> freeze, > > > >>>>> and > > > >>>>>>> there is a good reason for that, it should be explicitly > > discussed. > > > >> If > > > >>>>>>> there is no objection, it can be merged. > > > >>>>>>> > > > >>>>>>> Please let me know if you have a different understanding of > what > > > >>> feature > > > >>>>>>> freeze means. > > > >>>>>>> > > > >>>>>>> Regarding the issue of FLINK-13225 > > > >>>>>>> <https://issues.apache.org/jira/browse/FLINK-13225>? > > > >>>>>>> - Should we keep it? > > > >>>>>>> - Should we revert it in the release-1.9 branch and only > keep > > it > > > >>> for > > > >>>>>>> master? > > > >>>>>>> > > > >>>>>>> Best, > > > >>>>>>> Stephan > > > >>>>>>> > > > >>>>> > > > >>> > > > >>> > > > >> > > > > > > > > > > > > -- > > > > Xuefu Zhang > > > > > > > > "In Honey We Trust!" > > > > > > > > > -- Xuefu Zhang "In Honey We Trust!"
