Hi Xintong, We just found an issue FLINK-20285 and I think it is a blocker for 1.11.3.
Thanks, Zhu Gyula Fóra <gyula.f...@gmail.com> 于2020年11月18日周三 下午9:36写道： > Hi All! > > I have found the following issue today which might be considered a blocker > for this release as well: > > https://issues.apache.org/jira/browse/FLINK-20221 > > Could someone please quickly provide a second set of eyes and confirm that > this is indeed a big problem? :) > > Thank you! > Gyula > > On Wed, Nov 18, 2020 at 5:12 AM Xintong Song <tonysong...@gmail.com> > wrote: > > > Hi devs, > > > > Updates on the progress of preparing the 1.11.3 release. > > > > We are approaching the creation of our first release candidate. Thanks > all > > for the efforts so far. > > > > # Ongoing threads > > > > - *FLIP-27 backportings:* 74 out of 102 commits are already > backported. > > Stephan and Becket are actively working on this. > > - *Blockers:* There are 2 remaining blockers, both have been fixed on > > the master branch and should be backported along with the FLIP-27 > > changes. > > > > As soon as the above threads are done, we will create our first release > > candidate. If there's any other issues that you believe should be a > release > > blocker, please reach out to either me or Gordon. > > > > # JIRA version > > > > Version 1.11.4 has been created on JIRA. I've already tagged some of the > > tickets with the new fix version. Please feel free to continue with the > > efforts and merge once they are ready. I'll double check and update the > fix > > versions before the release. > > > > Thank you~ > > > > Xintong Song > > > > > > > > On Thu, Nov 12, 2020 at 1:31 PM Xintong Song <tonysong...@gmail.com> > > wrote: > > > > > Thanks for the notice and fix, Roman. > > > > > > Thank you~ > > > > > > Xintong Song > > > > > > > > > > > > On Wed, Nov 11, 2020 at 5:53 PM Khachatryan Roman < > > > khachatryan.ro...@gmail.com> wrote: > > > > > >> Hi, > > >> > > >> I'd like FLINK-20079 [1] to be merged into 1.11 and included in > 1.11.3. > > >> > > >> [1] https://issues.apache.org/jira/browse/FLINK-20079 > > >> > > >> Regards, > > >> Roman > > >> > > >> > > >> On Tue, Nov 10, 2020 at 8:21 AM Xintong Song <tonysong...@gmail.com> > > >> wrote: > > >> > > >> > Thanks for the notice, Dian. > > >> > > > >> > Thank you~ > > >> > > > >> > Xintong Song > > >> > > > >> > > > >> > > > >> > On Tue, Nov 10, 2020 at 10:19 AM Dian Fu <dian0511...@gmail.com> > > wrote: > > >> > > > >> > > Hi Xintong, > > >> > > > > >> > > I want to bring one more issue to your attention [1]. The test > case > > >> > > UnalignedCheckpointCompatibilityITCase.test failed several times > in > > >> the > > >> > > last nightly test of release-1.11. We need to figure out if it's > > just > > >> an > > >> > > instable test or caused by recent changes. > > >> > > > > >> > > [1] https://issues.apache.org/jira/browse/FLINK-20065 > > >> > > > > >> > > > 在 2020年11月10日，上午9:24，Xintong Song <tonysong...@gmail.com> 写道： > > >> > > > > > >> > > > Thanks for the replies. > > >> > > > > > >> > > > Thank you~ > > >> > > > > > >> > > > Xintong Song > > >> > > > > > >> > > > > > >> > > > > > >> > > > On Tue, Nov 10, 2020 at 1:09 AM Becket Qin < > becket....@gmail.com> > > >> > wrote: > > >> > > > > > >> > > >> Hi Xintong, > > >> > > >> > > >> > > >> Thanks for driving the release. Just want to sync up on the > > FLIP-27 > > >> > > >> backporting. Stephan and I are still trying to backport a bunch > > of > > >> > > patches > > >> > > >> of Source to 1.11.3. Including: > > >> > > >> > > >> > > >> [FLINK-19698][connector/common] Add a close() method to the > > >> > SplitReader. > > >> > > >> [FLINK-19717] SourceReaderBase.pollNext may return END_OF_INPUT > > if > > >> > > >> SplitReader.fetch throws > > >> > > >> [FLINK-19535] [connector/common] Avoid failing a job multiple > > >> times in > > >> > > >> SourceCoordinator. > > >> > > >> [FLINK-19265] [FLINK-20049][core] Source API final adjustments. > > >> > > >> > > >> > > >> and a few more fixes. > > >> > > >> > > >> > > >> We are currently trying to fix them in 1.12 first so it might > > take > > >> a > > >> > > little > > >> > > >> longer to backport them to 1.11.3. I think it will probably > take > > >> us a > > >> > > few > > >> > > >> more days to finish the backport. So that would roughly be the > > end > > >> of > > >> > > this > > >> > > >> week. > > >> > > >> > > >> > > >> Thanks, > > >> > > >> > > >> > > >> Jiangjie (Becket) Qin > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> On Mon, Nov 9, 2020 at 9:57 PM Till Rohrmann < > > trohrm...@apache.org > > >> > > > >> > > wrote: > > >> > > >> > > >> > > >>> Yes, I've downgraded FLINK-19816 to critical. > > >> > > >>> > > >> > > >>> Cheers, > > >> > > >>> Till > > >> > > >>> > > >> > > >>> On Mon, Nov 9, 2020 at 10:19 AM Xintong Song < > > >> tonysong...@gmail.com> > > >> > > >>> wrote: > > >> > > >>> > > >> > > >>>> Thanks for the notice, Till. > > >> > > >>>> > > >> > > >>>> I just checked and found FLINK-20033 is already fixed. Shall > we > > >> also > > >> > > >>>> downgrade FLINK-19816 to `Critical`? > > >> > > >>>> > > >> > > >>>> Thank you~ > > >> > > >>>> > > >> > > >>>> Xintong Song > > >> > > >>>> > > >> > > >>>> > > >> > > >>>> > > >> > > >>>> On Mon, Nov 9, 2020 at 4:42 PM Till Rohrmann < > > >> trohrm...@apache.org> > > >> > > >>> wrote: > > >> > > >>>> > > >> > > >>>>> I would like to bring one more critical issue to your > > attention > > >> > which > > >> > > >>> is > > >> > > >>>>> FLINK-20033 [1]. I believe that this issue is actually > causing > > >> what > > >> > > >> has > > >> > > >>>>> been reported in FLINK-19816 [2]. I hope to have it fixed by > > the > > >> > end > > >> > > >> of > > >> > > >>>>> today. Once FLINK-20033 is fixed, I think that we don't have > > to > > >> > block > > >> > > >>> the > > >> > > >>>>> release on FLINK-19816. > > >> > > >>>>> > > >> > > >>>>> [1] https://issues.apache.org/jira/browse/FLINK-20033 > > >> > > >>>>> [2] https://issues.apache.org/jira/browse/FLINK-19816 > > >> > > >>>>> > > >> > > >>>>> Cheers, > > >> > > >>>>> Till > > >> > > >>>>> > > >> > > >>>>> On Mon, Nov 9, 2020 at 4:05 AM Xintong Song < > > >> tonysong...@gmail.com > > >> > > > > >> > > >>>> wrote: > > >> > > >>>>> > > >> > > >>>>>> Hi devs, > > >> > > >>>>>> > > >> > > >>>>>> I'd like to provide an update on the progress of preparing > > >> release > > >> > > >>>>> 1.11.3. > > >> > > >>>>>> > > >> > > >>>>>> *Blockers* > > >> > > >>>>>> We currently have 3 remaining blockers. (3 resolved and 1 > > >> emerged > > >> > > >>>>> compared > > >> > > >>>>>> to last week) > > >> > > >>>>>> > > >> > > >>>>>> - [FLINK-19698] Add close() method and > onCheckpointComplete() > > >> to > > >> > > >> the > > >> > > >>>>>> Source. > > >> > > >>>>>> The issue has been fixed on the master branch. It's > currently > > >> > > >> blocked > > >> > > >>>> on > > >> > > >>>>>> the FLIP-27 backportings to backport it to the 1.11 branch. > > >> > > >>>>>> > > >> > > >>>>>> - [FLINK-19717] SourceReaderBase.pollNext may return > > >> END_OF_INPUT > > >> > > >> if > > >> > > >>>>>> SplitReader.fetch throws > > >> > > >>>>>> A PR has been opened and reviewed. From the discussions on > > the > > >> PR, > > >> > > >> it > > >> > > >>>>> looks > > >> > > >>>>>> close to mergeable. > > >> > > >>>>>> > > >> > > >>>>>> - [FLINK-19816] Flink restored from a wrong checkpoint (a > > very > > >> old > > >> > > >>> one > > >> > > >>>>> and > > >> > > >>>>>> not the last completed one) > > >> > > >>>>>> This is a newly emerged blocker and Matthias is working on > > it. > > >> > > >>>>>> > > >> > > >>>>>> *Test Instabilities* > > >> > > >>>>>> We currently have 27 test instabilities[1]. > > >> > > >>>>>> AFAIK, none of them are as serious as to block the 1.11.3 > > >> release. > > >> > > >>>>>> > > >> > > >>>>>> *FLIP-27 Backprotings* > > >> > > >>>>>> > > >> > > >>>>>> I noticed that there's no jira issues opened on the FLIP-27 > > >> > > >>> backporting > > >> > > >>>>>> efforts, which is part of the major efforts planned for the > > >> 1.11.3 > > >> > > >>>>> release, > > >> > > >>>>>> making it hard to track the progress. > > >> > > >>>>>> > > >> > > >>>>>> > > >> > > >>>>>> @Stephan and @Becket, could you please share the updates on > > the > > >> > > >>>>> backporting > > >> > > >>>>>> efforts? How is the progress and when are the efforts > > expected > > >> to > > >> > > >> be > > >> > > >>>>>> finished? It would be appreciated and helpful if we can > have > > a > > >> > jira > > >> > > >>>>> ticket > > >> > > >>>>>> to track the backporing progress publicly. > > >> > > >>>>>> > > >> > > >>>>>> > > >> > > >>>>>> Thank you~ > > >> > > >>>>>> > > >> > > >>>>>> Xintong Song > > >> > > >>>>>> > > >> > > >>>>>> > > >> > > >>>>>> [1] > > >> > > >>>>>> > > >> > > >>>>>> > > >> > > >>>>> > > >> > > >>>> > > >> > > >>> > > >> > > >> > > >> > > > > >> > > > >> > > > https://issues.apache.org/jira/browse/FLINK-16908?filter=12348580&jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20affectedVersion%20in%20(1.11.0%2C%201.11.1%2C%201.11.2%2C%201.11.3)%20AND%20labels%20%3D%20test-stability%20ORDER%20BY%20updated%20DESC%2C%20created%20DESC > > >> > > >>>>>> > > >> > > >>>>>> On Wed, Nov 4, 2020 at 3:58 PM Tzu-Li (Gordon) Tai < > > >> > > >>>> tzuli...@apache.org> > > >> > > >>>>>> wrote: > > >> > > >>>>>> > > >> > > >>>>>>> Thanks for the efforts so far with pushing for 1.11.3. > > >> > > >>>>>>> > > >> > > >>>>>>> I'd like to provide a mid-week update on how we're looking > > >> with > > >> > > >> all > > >> > > >>>> the > > >> > > >>>>>>> pending blockers and backports: > > >> > > >>>>>>> > > >> > > >>>>>>> *Blockers**:* > > >> > > >>>>>>> > > >> > > >>>>>>> Currently all blockers either have an open PR, or have > been > > >> > > >> merged. > > >> > > >>>>> I'll > > >> > > >>>>>>> highlight below the blockers *that still require reviewing > > >> > > >> efforts > > >> > > >>>> for > > >> > > >>>>>> them > > >> > > >>>>>>> to move forward:* > > >> > > >>>>>>> > > >> > > >>>>>>> - [FLINK-19909] Flink application in attach mode could not > > >> > > >>> terminate > > >> > > >>>>> when > > >> > > >>>>>>> the only job is canceled. PR: > > >> > > >>>>> https://github.com/apache/flink/pull/13911 > > >> > > >>>>>>> - [FLINK-19717] SourceReaderBase.pollNext may return > > >> END_OF_INPUT > > >> > > >>> if > > >> > > >>>>>>> SplitReader.fetch throws. PR: > > >> > > >>>>> https://github.com/apache/flink/pull/13776 > > >> > > >>>>>>> > > >> > > >>>>>>> The above PRs currently have no reviews at all yet. They > > seem > > >> to > > >> > > >>>>> already > > >> > > >>>>>>> have designated reviewers. > > >> > > >>>>>>> > > >> > > >>>>>>> > > >> > > >>>>>>> *Backports:* > > >> > > >>>>>>> - FLIP-27 robustness improvement backports: Stephan is > > >> currently > > >> > > >>>>> working > > >> > > >>>>>> on > > >> > > >>>>>>> backporting several FLIP-27 changes. There are no PRs yet > > for > > >> the > > >> > > >>>>>>> backported changes. > > >> > > >>>>>>> - DataStreamUtils.collect() refactorings backport: Steven > Wu > > >> > > >>>> mentioned > > >> > > >>>>> to > > >> > > >>>>>>> backport this, but AFAIK this isn't assigned to anyone > yet. > > >> > > >>>>>>> > > >> > > >>>>>>> *ETAs*: > > >> > > >>>>>>> > > >> > > >>>>>>> I'd like to request ETAs for the remaining backports, to > > >> prevent > > >> > > >> a > > >> > > >>>>> creep > > >> > > >>>>>> in > > >> > > >>>>>>> the scope of this bugfix release. > > >> > > >>>>>>> > > >> > > >>>>>>> We already have Flink users that would benefit from fixes > > that > > >> > > >> have > > >> > > >>>>> been > > >> > > >>>>>>> merged to release-1.11 already, so technically speaking > the > > >> > > >>> backports > > >> > > >>>>>>> should be considered "nice-to-have" (to the best of my > > >> knowledge > > >> > > >> of > > >> > > >>>> the > > >> > > >>>>>>> changes) and could potentially be moved to a follow-up > > 1.11.4. > > >> > > >>>>>>> Most notably, the Stateful Functions project is already > > >> waiting > > >> > > >> on > > >> > > >>>>> Flink > > >> > > >>>>>>> 1.11.3 to address critical recovery issues (please see the > > >> > > >> StateFun > > >> > > >>>>> 2.2.1 > > >> > > >>>>>>> release discussion thread [1]). > > >> > > >>>>>>> > > >> > > >>>>>>> @Stephan Ewen <se...@apache.org> @Becket Qin < > > >> > > >> becket....@gmail.com > > >> > > >>>> > > >> > > >>>>>> could > > >> > > >>>>>>> you provide an ETA for the FLIP-27 backports? It would > help > > to > > >> > > >> get > > >> > > >>> a > > >> > > >>>>>> better > > >> > > >>>>>>> estimate to decide how we proceed here. > > >> > > >>>>>>> > > >> > > >>>>>>> Cheers, > > >> > > >>>>>>> Gordon > > >> > > >>>>>>> > > >> > > >>>>>>> [1] > > >> > > >>>>>>> > > >> > > >>>>>>> > > >> > > >>>>>> > > >> > > >>>>> > > >> > > >>>> > > >> > > >>> > > >> > > >> > > >> > > > > >> > > > >> > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-StateFun-hotfix-version-2-2-1-td46239.html > > >> > > >>>>>>> > > >> > > >>>>>>> On Wed, Nov 4, 2020 at 3:16 PM Tzu-Li (Gordon) Tai < > > >> > > >>>>> tzuli...@apache.org> > > >> > > >>>>>>> wrote: > > >> > > >>>>>>> > > >> > > >>>>>>>>> The collect() utils can be picked back, I see no issue > > with > > >> > > >>> that > > >> > > >>>>> (it > > >> > > >>>>>> is > > >> > > >>>>>>>> isolated utilities). > > >> > > >>>>>>>> > > >> > > >>>>>>>> Just checking on all the requested backports mentioned in > > >> this > > >> > > >>>>> thread, > > >> > > >>>>>>> and > > >> > > >>>>>>>> figuring out which ones seem to still be unassigned / > open. > > >> > > >>>>>>>> > > >> > > >>>>>>>> Is someone working on backporting > > >> > > >>>>>>>> > > >> > > >>>>>>> > > >> > > >>>>>> > > >> > > >>>>> > > >> > > >>>> > > >> > > >>> > > >> > > >> > > >> > > > > >> > > > >> > > > https://github.com/apache/flink/commit/09a7a66b7313fea64817fe960a8da1265b428efc > > >> > > >>>>>>>> to release-1.11 at the moment? > > >> > > >>>>>>>> > > >> > > >>>>>>>> On Wed, Nov 4, 2020 at 1:39 AM Steven Wu < > > >> stevenz...@gmail.com > > >> > > >>> > > >> > > >>>>> wrote: > > >> > > >>>>>>>> > > >> > > >>>>>>>>> @Stephan Ewen <se...@apache.org> yeah, we can do that. > > >> don't > > >> > > >>>> worry > > >> > > >>>>>>> about > > >> > > >>>>>>>>> it. your earlier email had the perfect explanation on > why > > >> file > > >> > > >>>>> source > > >> > > >>>>>>>>> shouldn't be backported. > > >> > > >>>>>>>>> > > >> > > >>>>>>>>> On Tue, Nov 3, 2020 at 3:37 AM Stephan Ewen < > > >> se...@apache.org > > >> > > >>> > > >> > > >>>>> wrote: > > >> > > >>>>>>>>> > > >> > > >>>>>>>>>> @Steven would it be possible to initially copy some of > > the > > >> > > >>> code > > >> > > >>>>> into > > >> > > >>>>>>> the > > >> > > >>>>>>>>>> iceberg source and later replace it by a dependency on > > the > > >> > > >>> Flink > > >> > > >>>>>> file > > >> > > >>>>>>>>>> source? > > >> > > >>>>>>>>>> > > >> > > >>>>>>>>>> On Mon, Nov 2, 2020 at 8:33 PM Steven Wu < > > >> > > >>> stevenz...@gmail.com> > > >> > > >>>>>>> wrote: > > >> > > >>>>>>>>>> > > >> > > >>>>>>>>>>> Stephan, thanks a lot for explaining the file > connector. > > >> > > >>> that > > >> > > >>>>>> makes > > >> > > >>>>>>>>>> sense. > > >> > > >>>>>>>>>>> > > >> > > >>>>>>>>>>> I was asking because we were trying to reuse some of > the > > >> > > >>>>>>>>> implementations > > >> > > >>>>>>>>>> in > > >> > > >>>>>>>>>>> the file source for Iceberg source. Flink Iceberg > source > > >> > > >>> lives > > >> > > >>>>> in > > >> > > >>>>>>> the > > >> > > >>>>>>>>>>> Iceberg repo, which is not possible to code against > the > > >> > > >>> master > > >> > > >>>>>>> branch > > >> > > >>>>>>>>> of > > >> > > >>>>>>>>>>> the Flink code. > > >> > > >>>>>>>>>>> > > >> > > >>>>>>>>>>> On Mon, Nov 2, 2020 at 3:31 AM Stephan Ewen < > > >> > > >>> se...@apache.org > > >> > > >>>>> > > >> > > >>>>>>> wrote: > > >> > > >>>>>>>>>>> > > >> > > >>>>>>>>>>>> Hi Steven! > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>>> So far there are no plans to pick back the file > system > > >> > > >>>>> connector > > >> > > >>>>>>>>> code. > > >> > > >>>>>>>>>>> This > > >> > > >>>>>>>>>>>> is still evolving and not finalized for 1.12, so I > > don't > > >> > > >>>> feel > > >> > > >>>>> it > > >> > > >>>>>>> is > > >> > > >>>>>>>>> a > > >> > > >>>>>>>>>>> good > > >> > > >>>>>>>>>>>> candidate to be backported. > > >> > > >>>>>>>>>>>> However, with the base connector changes backported, > > you > > >> > > >>>>> should > > >> > > >>>>>> be > > >> > > >>>>>>>>> able > > >> > > >>>>>>>>>>> to > > >> > > >>>>>>>>>>>> run the file connector code from master against > 1.11.3. > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>>> The collect() utils can be picked back, I see no > issue > > >> > > >>> with > > >> > > >>>>> that > > >> > > >>>>>>>>> (it is > > >> > > >>>>>>>>>>>> isolated utilities). > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>>> Best, > > >> > > >>>>>>>>>>>> Stephan > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>>> On Mon, Nov 2, 2020 at 3:02 AM Steven Wu < > > >> > > >>>>> stevenz...@gmail.com> > > >> > > >>>>>>>>> wrote: > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>>>> Basically, it would be great to get the latest code > in > > >> > > >>> the > > >> > > >>>>>>>>>>>>> flink-connector-files (FLIP-27). > > >> > > >>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>> On Sat, Oct 31, 2020 at 9:57 AM Steven Wu < > > >> > > >>>>>> stevenz...@gmail.com > > >> > > >>>>>>>> > > >> > > >>>>>>>>>>> wrote: > > >> > > >>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>> Stephan, it will be great if we can also backport > > >> > > >> the > > >> > > >>>>>>>>>> DataStreamUtils > > >> > > >>>>>>>>>>>>>> related commits that help with collecting output > > >> > > >> from > > >> > > >>>>>>> unbounded > > >> > > >>>>>>>>>>>> streams. > > >> > > >>>>>>>>>>>>>> e.g. > > >> > > >>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>> > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>> > > >> > > >>>>>>>>>> > > >> > > >>>>>>>>> > > >> > > >>>>>>> > > >> > > >>>>>> > > >> > > >>>>> > > >> > > >>>> > > >> > > >>> > > >> > > >> > > >> > > > > >> > > > >> > > > https://github.com/apache/flink/commit/09a7a66b7313fea64817fe960a8da1265b428efc > > >> > > >>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>> I tried to copy and paste the code to unblock > > >> > > >> myself. > > >> > > >>>> but > > >> > > >>>>> it > > >> > > >>>>>>>>>> quickly > > >> > > >>>>>>>>>>>> got > > >> > > >>>>>>>>>>>>>> into the rabbit hole of more and more code. > > >> > > >>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>> On Fri, Oct 30, 2020 at 11:02 AM Stephan Ewen < > > >> > > >>>>>>> se...@apache.org > > >> > > >>>>>>>>>> > > >> > > >>>>>>>>>>>> wrote: > > >> > > >>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>> I have started with backporting the source API > > >> > > >>> changes. > > >> > > >>>>>> Some > > >> > > >>>>>>>>> minor > > >> > > >>>>>>>>>>>>>>> conflicts to solve, will need a bit more to finish > > >> > > >>>> this. > > >> > > >>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>> On Fri, Oct 30, 2020 at 7:25 AM Tzu-Li (Gordon) > > >> > > >> Tai < > > >> > > >>>>>>>>>>>>> tzuli...@apache.org> > > >> > > >>>>>>>>>>>>>>> wrote: > > >> > > >>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>> @Stephan Ewen <se...@apache.org> > > >> > > >>>>>>>>>>>>>>>> Are there already plans or ongoing efforts for > > >> > > >>>>>> backporting > > >> > > >>>>>>>>> the > > >> > > >>>>>>>>>>> list > > >> > > >>>>>>>>>>>> of > > >> > > >>>>>>>>>>>>>>>> FLIP-27 changes that you posted? > > >> > > >>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>> On Thu, Oct 29, 2020 at 7:08 PM Xintong Song < > > >> > > >>>>>>>>>>> tonysong...@gmail.com > > >> > > >>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>> wrote: > > >> > > >>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>> Hi folks, > > >> > > >>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>> Just to provide some updates concerning the > > >> > > >> status > > >> > > >>>> on > > >> > > >>>>>> the > > >> > > >>>>>>>>>>>>>>>>> test instabilities. > > >> > > >>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>> Currently, we have 30 unresolved tickets labeled > > >> > > >>>> with > > >> > > >>>>>>>>> `Affects > > >> > > >>>>>>>>>>>>> Version` > > >> > > >>>>>>>>>>>>>>>>> 1.11.x. > > >> > > >>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>> > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>> > > >> > > >>>>>>>>>> > > >> > > >>>>>>>>> > > >> > > >>>>>>> > > >> > > >>>>>> > > >> > > >>>>> > > >> > > >>>> > > >> > > >>> > > >> > > >> > > >> > > > > >> > > > >> > > > https://issues.apache.org/jira/browse/FLINK-19775?filter=12348580&jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20affectedVersion%20in%20(1.11.0%2C%201.11.1%2C%201.11.2%2C%201.11.3)%20AND%20labels%20%3D%20test-stability%20ORDER%20BY%20created%20DESC > > >> > > >>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>> Among the 30 tickets, 11 of them are: > > >> > > >>>>>>>>>>>>>>>>> - Have occured in the recent 3 months > > >> > > >>>>>>>>>>>>>>>>> - Not confirmed to be pure testability issues > > >> > > >>>>>>>>>>>>>>>>> - Not confirmed to be rare condition cases > > >> > > >>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>> It would be nice if someone familiar with these > > >> > > >>>>>> components > > >> > > >>>>>>>>> can > > >> > > >>>>>>>>>>>> take a > > >> > > >>>>>>>>>>>>>>> look > > >> > > >>>>>>>>>>>>>>>>> into these issues. > > >> > > >>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>> - > > >> > > >>> https://issues.apache.org/jira/browse/FLINK-17159 > > >> > > >>>>>> (ES6) > > >> > > >>>>>>>>>>>>>>>>> - > > >> > > >>> https://issues.apache.org/jira/browse/FLINK-17912 > > >> > > >>>>>>> (Kafka) > > >> > > >>>>>>>>>>>>>>>>> - > > >> > > >>> https://issues.apache.org/jira/browse/FLINK-17949 > > >> > > >>>>>>> (Kafka) > > >> > > >>>>>>>>>>>>>>>>> ⁃ > > >> > > >>> https://issues.apache.org/jira/browse/FLINK-18444 > > >> > > >>>>>>> (Kafka) > > >> > > >>>>>>>>>>>>>>>>> - > > >> > > >>> https://issues.apache.org/jira/browse/FLINK-18634 > > >> > > >>>>>>> (Kafka) > > >> > > >>>>>>>>>>>>>>>>> - > > >> > > >>> https://issues.apache.org/jira/browse/FLINK-18648 > > >> > > >>>>>>> (Kafka) > > >> > > >>>>>>>>>>>>>>>>> - > > >> > > >>> https://issues.apache.org/jira/browse/FLINK-18807 > > >> > > >>>>>>> (Kafka) > > >> > > >>>>>>>>>>>>>>>>> - > > >> > > >>> https://issues.apache.org/jira/browse/FLINK-19369 > > >> > > >>>>>>>>>>>> (BlobClientTest) > > >> > > >>>>>>>>>>>>>>>>> - > > >> > > >>> https://issues.apache.org/jira/browse/FLINK-19436 > > >> > > >>>>>>> (TPCDS) > > >> > > >>>>>>>>>>>>>>>>> - > > >> > > >>> https://issues.apache.org/jira/browse/FLINK-19690 > > >> > > >>>>>>>>>>>> (Format/Parquet) > > >> > > >>>>>>>>>>>>>>>>> - > > >> > > >>> https://issues.apache.org/jira/browse/FLINK-19775 > > >> > > >>>>>>>>>>>>>>>>> (SystemProcessingTimeServiceTest) > > >> > > >>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>> Thank you~ > > >> > > >>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>> Xintong Song > > >> > > >>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>> On Thu, Oct 29, 2020 at 10:21 AM Jingsong Li < > > >> > > >>>>>>>>>>>> jingsongl...@gmail.com > > >> > > >>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>> wrote: > > >> > > >>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>> +1 to backport the FLIP-27 adjustments to > > >> > > >>> 1.11.x. > > >> > > >>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>> If possible, that would be great. Many people > > >> > > >>> are > > >> > > >>>>>>> looking > > >> > > >>>>>>>>>>> forward > > >> > > >>>>>>>>>>>>> to > > >> > > >>>>>>>>>>>>>>> the > > >> > > >>>>>>>>>>>>>>>>>> FLIP-27 interface, but they don't want to take > > >> > > >>> the > > >> > > >>>>>> risk > > >> > > >>>>>>> to > > >> > > >>>>>>>>>>>> upgrade > > >> > > >>>>>>>>>>>>> to > > >> > > >>>>>>>>>>>>>>>>> 1.12 > > >> > > >>>>>>>>>>>>>>>>>> (And wait 1.12). After all, 1.11 is a > > >> > > >> relatively > > >> > > >>>>>> stable > > >> > > >>>>>>>>>>> version. > > >> > > >>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>> Best, > > >> > > >>>>>>>>>>>>>>>>>> Jingsong > > >> > > >>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>> On Thu, Oct 29, 2020 at 1:24 AM Stephan Ewen < > > >> > > >>>>>>>>>> se...@apache.org > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>> wrote: > > >> > > >>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> Thanks for starting this. > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> +1 form my side to backport the FLIP-27 > > >> > > >>>>> adjustments > > >> > > >>>>>> to > > >> > > >>>>>>>>>>> 1.11.x. > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> There were quite a few changes, and I think > > >> > > >> we > > >> > > >>>>> need > > >> > > >>>>>> to > > >> > > >>>>>>>>>>>>> cherry-pick > > >> > > >>>>>>>>>>>>>>>>> them > > >> > > >>>>>>>>>>>>>>>>>> all > > >> > > >>>>>>>>>>>>>>>>>>> to not get some inconsistent mix of changes > > >> > > >>> and > > >> > > >>>>> many > > >> > > >>>>>>>>> merge > > >> > > >>>>>>>>>>>>>>> conflicts. > > >> > > >>>>>>>>>>>>>>>>>>> I made a list below of what we need to add > > >> > > >> to > > >> > > >>>>>>>>>> "release-1.11". > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> *#### Core Source API Changes to backport > > >> > > >> (in > > >> > > >>>>>> REVERSE > > >> > > >>>>>>>>>> order)* > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> (Use: "git log > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>> > > >> > > >>>>> > > flink-core/src/main/java/org/apache/flink/api/connector/source") > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> 162c072e9265a7b6dd9d6f5459eb7974091c4c4e > > >> > > >>>>>>>>>>>>>>>>>>> [FLINK-19492][core] Consolidate Source > > >> > > >> Events > > >> > > >>>>>> between > > >> > > >>>>>>>>>> Source > > >> > > >>>>>>>>>>>> API > > >> > > >>>>>>>>>>>>>>> and > > >> > > >>>>>>>>>>>>>>>>>> Split > > >> > > >>>>>>>>>>>>>>>>>>> Reader API > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> 428fbbad85379f71a9fbf1a1ab3e3dcfa9006436 > > >> > > >>>>>>>>>>>>>>>>>>> [FLINK-19205][core] Add access to > > >> > > >>> configuration > > >> > > >>>>> and > > >> > > >>>>>>>>>> hostname > > >> > > >>>>>>>>>>> in > > >> > > >>>>>>>>>>>>> the > > >> > > >>>>>>>>>>>>>>>>>>> SourceReaderContext > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> 723e1790744ddc78d19c7c978442af1383f38d33 > > >> > > >>>>>>>>>>>>>>>>>>> [hotfix][core] Add to Source Enumerator > > >> > > >>>>> convenience > > >> > > >>>>>>>>> methods > > >> > > >>>>>>>>>>> to > > >> > > >>>>>>>>>>>>>>> assign > > >> > > >>>>>>>>>>>>>>>>>>> single split > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> *#### Connector Base Changes to Backport (in > > >> > > >>>>> REVERSE > > >> > > >>>>>>>>>> order)* > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> (Use: "git log > > >> > > >>>>>>> flink-connectors/flink-connector-base") > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> 401f56fe9d6b0271260edf9787cdcbfe4d03874d > > >> > > >>>>>>>>>>>>>>>>>>> [FLINK-19427][FLINK-19489][tests] Fix test > > >> > > >>>>>> conditions > > >> > > >>>>>>>>> for > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>> 'SplitFetcherTest.testNotifiesWhenGoingIdleConcurrent()' > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> 68c5c2ff779d82a1ff81ffaf60d8a1b283797db1 > > >> > > >>>>>>>>>>>>>>>>>>> [FLINK-19448][connector base] Explicitly > > >> > > >> check > > >> > > >>>> for > > >> > > >>>>>>>>>>> un-expected > > >> > > >>>>>>>>>>>>>>>>> condition > > >> > > >>>>>>>>>>>>>>>>>>> that would leave an inconsistent state > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> 162c072e9265a7b6dd9d6f5459eb7974091c4c4e > > >> > > >>>>>>>>>>>>>>>>>>> [FLINK-19492][core] Consolidate Source > > >> > > >> Events > > >> > > >>>>>> between > > >> > > >>>>>>>>>> Source > > >> > > >>>>>>>>>>>> API > > >> > > >>>>>>>>>>>>>>> and > > >> > > >>>>>>>>>>>>>>>>>> Split > > >> > > >>>>>>>>>>>>>>>>>>> Reader API > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> c1ca7a4c7c21ec8868c14cdddd43c559625b794c > > >> > > >>>>>>>>>>>>>>>>>>> [refactor][tests] Move some source test > > >> > > >> utils > > >> > > >>>> from > > >> > > >>>>>>>>>>>>>>>>> flink-connector-base > > >> > > >>>>>>>>>>>>>>>>>> to > > >> > > >>>>>>>>>>>>>>>>>>> flink-core > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> ee5c4c211c35c70d28252363bbc8400453609977 > > >> > > >>>>>>>>>>>>>>>>>>> [FLINK-19251][connectors] Avoid confusing > > >> > > >>> queue > > >> > > >>>>>>>>> handling in > > >> > > >>>>>>>>>>>>>>>>>>> "SplitReader.handleSplitsChanges()" > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> 5abef56b2bf85bcac786f6b16b6899b6cced7176 > > >> > > >>>>>>>>>>>>>>>>>>> [FLINK-19250][connectors] Fix error > > >> > > >>> propagation > > >> > > >>>> in > > >> > > >>>>>>>>>> connector > > >> > > >>>>>>>>>>>> base > > >> > > >>>>>>>>>>>>>>>>>>> (SplitFetcherManager). > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> 8fcca837c55a9216595ee4c03038b52747098dbb > > >> > > >>>>>>>>>>>>>>>>>>> [hotfix][connectors] Improve JavaDocs for > > >> > > >>>>>>>>>>>>>>> SingleThreadFetcherManager > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> 4700bb5dde3303cbe98882f6beb7379425717b01 > > >> > > >>>>>>>>>>>>>>>>>>> [FLINK-19225][connectors] Various small > > >> > > >>>>> improvements > > >> > > >>>>>>> to > > >> > > >>>>>>>>>>>>>>>>> SourceReaderBase > > >> > > >>>>>>>>>>>>>>>>>>> (part 2) > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> 12261c6b7ed6478a9b9f6a69cb58246b83cab9b7 > > >> > > >>>>>>>>>>>>>>>>>>> [FLINK-17393][connectors] (follow-up) Wakeup > > >> > > >>> the > > >> > > >>>>>>>>>>> SplitFetchers > > >> > > >>>>>>>>>>>>> more > > >> > > >>>>>>>>>>>>>>>>>>> elegantly. > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> c60aaff0249bfd6b5871b7f82e03efc487a54d6b > > >> > > >>>>>>>>>>>>>>>>>>> [hotfix][tests] Extend test coverage for > > >> > > >>>>>>>>>>>>>>> FutureCompletingBlockingQueue > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> cef8a587d7fd2fe64cc644da5ed095d82e46f631 > > >> > > >>>>>>>>>>>>>>>>>>> [FLINK-19245][connectors] Set default > > >> > > >> capacity > > >> > > >>>> for > > >> > > >>>>>>>>>>>>>>>>>>> FutureCompletingBlockingQueue. > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> 4ea95782b4c6a2538153d4d16ad3f4839c7de0fb > > >> > > >>>>>>>>>>>>>>>>>>> [FLINK-19223][connectors] Simplify > > >> > > >>> Availability > > >> > > >>>>>> Future > > >> > > >>>>>>>>>> Model > > >> > > >>>>>>>>>>> in > > >> > > >>>>>>>>>>>>>>> Base > > >> > > >>>>>>>>>>>>>>>>>>> Connector > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> 511857049ba30c8ff0ee56da551fa4a479dc583e > > >> > > >>>>>>>>>>>>>>>>>>> [FLINK-18128][connectors] Ensure idle split > > >> > > >>>>> fetchers > > >> > > >>>>>>>>> lead > > >> > > >>>>>>>>>> to > > >> > > >>>>>>>>>>>>>>>>> availability > > >> > > >>>>>>>>>>>>>>>>>>> notifications. > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> a8206467af0830dcb89623ea068b5ca3b3450c92 > > >> > > >>>>>>>>>>>>>>>>>>> [refactor][core] Eagerly initialize the > > >> > > >>>> FetchTask > > >> > > >>>>> to > > >> > > >>>>>>>>>> support > > >> > > >>>>>>>>>>>>> proper > > >> > > >>>>>>>>>>>>>>>>> unit > > >> > > >>>>>>>>>>>>>>>>>>> testing > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> 3b2f54bcb437f98e6137c904045cc51072b5c06b > > >> > > >>>>>>>>>>>>>>>>>>> [hotfix][tests] Move constants in > > >> > > >>>> SplitFetcherTest > > >> > > >>>>>>>>> relevant > > >> > > >>>>>>>>>>> to > > >> > > >>>>>>>>>>>>> only > > >> > > >>>>>>>>>>>>>>>>> one > > >> > > >>>>>>>>>>>>>>>>>>> test into test method > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> d7625760a75a508bf05bcddc380bb4d62ee1743e > > >> > > >>>>>>>>>>>>>>>>>>> [FLINK-19225][connectors] Various small > > >> > > >>>>> improvements > > >> > > >>>>>>> to > > >> > > >>>>>>>>>>>>>>>>> SourceReaderBase > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> a5b0d3297748c1be47ad579a88f24df2255a8df1 > > >> > > >>>>>>>>>>>>>>>>>>> [FLINK-17393][connectors] Wakeup the > > >> > > >>>> SplitFetchers > > >> > > >>>>>>> more > > >> > > >>>>>>>>>>>>> elegantly. > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> f42a3ebc3e81a034b7221a803c153636fef34903 > > >> > > >>>>>>>>>>>>>>>>>>> [FLINK-18680][connectors] Make connector > > >> > > >> base > > >> > > >>>>>>>>>>>> RecordsWithSplitIds > > >> > > >>>>>>>>>>>>>>> more > > >> > > >>>>>>>>>>>>>>>>>>> lightweight. > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> e3d273de822b085183d09b275a445879ff94b350 > > >> > > >>>>>>>>>>>>>>>>>>> [FLINK-19162][connectors] Add 'recycle()' to > > >> > > >>> the > > >> > > >>>>>>>>>>>>>>> RecordsWithSplitIds > > >> > > >>>>>>>>>>>>>>>>> to > > >> > > >>>>>>>>>>>>>>>>>>> support reuse of heavy objects. > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> 8ebc464c2520453a70001cd712abc8dee6ee89e0 > > >> > > >>>>>>>>>>>>>>>>>>> [hotfix][testing] Add a set of > > >> > > >> parameterizable > > >> > > >>>>>> testing > > >> > > >>>>>>>>>> mocks > > >> > > >>>>>>>>>>>> for > > >> > > >>>>>>>>>>>>>>> the > > >> > > >>>>>>>>>>>>>>>>>> Split > > >> > > >>>>>>>>>>>>>>>>>>> Reader API > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> 930a07438be1185388d7150640f294dfe2a5d378 > > >> > > >>>>>>>>>>>>>>>>>>> [hotfix][connectors] Add RequestSplitEvent > > >> > > >> to > > >> > > >>>>>>>>>>>>>>> 'flink-connector-base' > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> commit > > >> > > >>> e945ce8a933bc378844782f784ca473c767ca159 > > >> > > >>>>>>>>>>>>>>>>>>> [refactor][connectors] Backport of the > > >> > > >>>>>> connector-base > > >> > > >>>>>>>>>>> exception > > >> > > >>>>>>>>>>>>>>>>> handling > > >> > > >>>>>>>>>>>>>>>>>>> from the Kafka Connector Pull Request > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> On Wed, Oct 28, 2020 at 5:49 PM Steven Wu < > > >> > > >>>>>>>>>>>> stevenz...@gmail.com> > > >> > > >>>>>>>>>>>>>>>>> wrote: > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>> I would love to see this FLIP-27 source > > >> > > >>>>> interface > > >> > > >>>>>>>>>>> improvement > > >> > > >>>>>>>>>>>>> [1] > > >> > > >>>>>>>>>>>>>>>>> made > > >> > > >>>>>>>>>>>>>>>>>> to > > >> > > >>>>>>>>>>>>>>>>>>>> 1.11.3. > > >> > > >>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>> [1] > > >> > > >>>>>>> https://issues.apache.org/jira/browse/FLINK-19698 > > >> > > >>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>> On Wed, Oct 28, 2020 at 12:32 AM Tzu-Li > > >> > > >>>> (Gordon) > > >> > > >>>>>>> Tai < > > >> > > >>>>>>>>>>>>>>>>>>> tzuli...@apache.org> > > >> > > >>>>>>>>>>>>>>>>>>>> wrote: > > >> > > >>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>> Thanks for the replies so far! > > >> > > >>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>> Just to provide a brief update on the > > >> > > >>> status > > >> > > >>>>> of > > >> > > >>>>>>>>>> blockers > > >> > > >>>>>>>>>>>> for > > >> > > >>>>>>>>>>>>>>>>> 1.11.3 > > >> > > >>>>>>>>>>>>>>>>>> so > > >> > > >>>>>>>>>>>>>>>>>>>> far: > > >> > > >>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>> *PR opened, pending reviewer*- > > >> > > >>> [FLINK-19717] > > >> > > >>>>>>>>>>>>>>>>>> SourceReaderBase.pollNext > > >> > > >>>>>>>>>>>>>>>>>>>> may > > >> > > >>>>>>>>>>>>>>>>>>>>> return END_OF_INPUT if SplitReader.fetch > > >> > > >>>>> throws > > >> > > >>>>>> ( > > >> > > >>>>>>>>>>>>>>>>>>>>> > > >> > > >>> https://github.com/apache/flink/pull/13776) > > >> > > >>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>> *PR opened, reviewed + close to being > > >> > > >>>> merged* > > >> > > >>>>>>>>>>>>>>>>>>>>> - [FLINK-19741] Timer service should > > >> > > >> skip > > >> > > >>>>>>> restoring > > >> > > >>>>>>>>>> from > > >> > > >>>>>>>>>>>> raw > > >> > > >>>>>>>>>>>>>>> keyed > > >> > > >>>>>>>>>>>>>>>>>>> stream > > >> > > >>>>>>>>>>>>>>>>>>>>> if it isn't the writer ( > > >> > > >>>>>>>>>>>>>>>>> https://github.com/apache/flink/pull/13761) > > >> > > >>>>>>>>>>>>>>>>>>>>> - [FLINK-19748] Raw keyed stream key > > >> > > >> group > > >> > > >>>>>>> iterator > > >> > > >>>>>>>>>>> should > > >> > > >>>>>>>>>>>> be > > >> > > >>>>>>>>>>>>>>>>>> skipping > > >> > > >>>>>>>>>>>>>>>>>>>>> unwritten key groups ( > > >> > > >>>>>>>>>>>>>>> https://github.com/apache/flink/pull/13772) > > >> > > >>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>> *Merged* > > >> > > >>>>>>>>>>>>>>>>>>>>> - [FLINK-19154] Application mode deletes > > >> > > >>> HA > > >> > > >>>>> data > > >> > > >>>>>>> in > > >> > > >>>>>>>>>> case > > >> > > >>>>>>>>>>> of > > >> > > >>>>>>>>>>>>>>>>> suspended > > >> > > >>>>>>>>>>>>>>>>>>>>> ZooKeeper connection > > >> > > >>>>>>>>>>>>>>>>>>>>> - [FLINK-19569] Upgrade ICU4J to 67.1+ > > >> > > >>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>> Right now as it seems, progress is > > >> > > >> mainly > > >> > > >>>>>> blocked > > >> > > >>>>>>>>> on a > > >> > > >>>>>>>>>>>>> reviewer > > >> > > >>>>>>>>>>>>>>>>> for > > >> > > >>>>>>>>>>>>>>>>>>>>> FLINK-19717. > > >> > > >>>>>>>>>>>>>>>>>>>>> Meanwhile, Xintong is keeping an eye on > > >> > > >>> test > > >> > > >>>>>>>>>>> instabilities > > >> > > >>>>>>>>>>>>> [1] > > >> > > >>>>>>>>>>>>>>> to > > >> > > >>>>>>>>>>>>>>>>> see > > >> > > >>>>>>>>>>>>>>>>>>> if > > >> > > >>>>>>>>>>>>>>>>>>>>> there are any fixes that should be > > >> > > >> applied > > >> > > >>>> to > > >> > > >>>>>>>>>>>> `release-1.11`. > > >> > > >>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>> This is also a reminder, that if there > > >> > > >> are > > >> > > >>>>> other > > >> > > >>>>>>>>>> blockers > > >> > > >>>>>>>>>>>>> that > > >> > > >>>>>>>>>>>>>>> we > > >> > > >>>>>>>>>>>>>>>>>> need > > >> > > >>>>>>>>>>>>>>>>>>> to > > >> > > >>>>>>>>>>>>>>>>>>>>> be aware of, or a need to re-establish > > >> > > >>>>> estimated > > >> > > >>>>>>>>> time > > >> > > >>>>>>>>>> for > > >> > > >>>>>>>>>>>>>>> getting > > >> > > >>>>>>>>>>>>>>>>>> fixes > > >> > > >>>>>>>>>>>>>>>>>>>> in > > >> > > >>>>>>>>>>>>>>>>>>>>> and delay the RC for 1.11.3, please do > > >> > > >> let > > >> > > >>>> us > > >> > > >>>>>>> know! > > >> > > >>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>> Cheers, > > >> > > >>>>>>>>>>>>>>>>>>>>> Gordon > > >> > > >>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>> [1] > > >> > > >>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>> > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>> > > >> > > >>>>>>>>>> > > >> > > >>>>>>>>> > > >> > > >>>>>>> > > >> > > >>>>>> > > >> > > >>>>> > > >> > > >>>> > > >> > > >>> > > >> > > >> > > >> > > > > >> > > > >> > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20affectedVersion%20in%20(1.11.0%2C%201.11.1%2C%201.11.2)%20AND%20labels%20%3D%20test-stability > > >> > > >>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>> On Mon, Oct 26, 2020 at 9:43 PM Kostas > > >> > > >>>>> Kloudas < > > >> > > >>>>>>>>>>>>>>>>> kklou...@apache.org> > > >> > > >>>>>>>>>>>>>>>>>>>>> wrote: > > >> > > >>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>> +1 for releasing Flink 1.11.3 as it > > >> > > >>>>> contains a > > >> > > >>>>>>>>> number > > >> > > >>>>>>>>>>> of > > >> > > >>>>>>>>>>>>>>>>> important > > >> > > >>>>>>>>>>>>>>>>>>>>>> fixes and thanks Gordon and Xintong > > >> > > >> for > > >> > > >>>>>>>>> volunteering. > > >> > > >>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>> Cheers, > > >> > > >>>>>>>>>>>>>>>>>>>>>> Kostas > > >> > > >>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>> On Mon, Oct 26, 2020 at 4:37 AM Yu Li > > >> > > >> < > > >> > > >>>>>>>>>>> car...@gmail.com> > > >> > > >>>>>>>>>>>>>>> wrote: > > >> > > >>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>> +1 for releasing Flink 1.11.3, and > > >> > > >>>> thanks > > >> > > >>>>>>> Gordon > > >> > > >>>>>>>>>> and > > >> > > >>>>>>>>>>>>>>> Xintong > > >> > > >>>>>>>>>>>>>>>>> for > > >> > > >>>>>>>>>>>>>>>>>>>>>>> volunteering as our release > > >> > > >> managers. > > >> > > >>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>> Best Regards, > > >> > > >>>>>>>>>>>>>>>>>>>>>>> Yu > > >> > > >>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>> On Mon, 26 Oct 2020 at 09:45, > > >> > > >> Xintong > > >> > > >>>>> Song < > > >> > > >>>>>>>>>>>>>>>>>> tonysong...@gmail.com> > > >> > > >>>>>>>>>>>>>>>>>>>>>> wrote: > > >> > > >>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> Thanks Gordan for starting this > > >> > > >>>>>> discussion. > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> My pleasure to help with the > > >> > > >> release > > >> > > >>>>>>> process. > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> Thank you~ > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> Xintong Song > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> On Fri, Oct 23, 2020 at 11:29 PM > > >> > > >>> Till > > >> > > >>>>>>>>> Rohrmann < > > >> > > >>>>>>>>>>>>>>>>>>>> trohrm...@apache.org > > >> > > >>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> wrote: > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks for starting this > > >> > > >>> discussion > > >> > > >>>>>>> Gordon. > > >> > > >>>>>>>>>> There > > >> > > >>>>>>>>>>>> are > > >> > > >>>>>>>>>>>>>>> over > > >> > > >>>>>>>>>>>>>>>>>> 100 > > >> > > >>>>>>>>>>>>>>>>>>>>> issues > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> which are fixed for 1.11.3. > > >> > > >> Hence > > >> > > >>> +1 > > >> > > >>>>>> for a > > >> > > >>>>>>>>>>> soonish > > >> > > >>>>>>>>>>>>>>> 1.11.3 > > >> > > >>>>>>>>>>>>>>>>>>>> release. > > >> > > >>>>>>>>>>>>>>>>>>>>>> Thanks > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> for volunteering as our release > > >> > > >>>>> managers > > >> > > >>>>>>>>> Gordon > > >> > > >>>>>>>>>>> and > > >> > > >>>>>>>>>>>>>>>>> Xintong! > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Cheers, > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Till > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Oct 23, 2020 at 5:02 PM > > >> > > >>>> Tzu-Li > > >> > > >>>>>>>>> (Gordon) > > >> > > >>>>>>>>>>>> Tai < > > >> > > >>>>>>>>>>>>>>>>>>>>>> tzuli...@apache.org > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote: > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Hi, > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Xintong and I would like to > > >> > > >>> start a > > >> > > >>>>>>>>> discussion > > >> > > >>>>>>>>>>> for > > >> > > >>>>>>>>>>>>>>>>> releasing > > >> > > >>>>>>>>>>>>>>>>>>>> Flink > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> 1.11.3 > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> soon. > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> It seems like we already have a > > >> > > >>> few > > >> > > >>>>>>>>> pressing > > >> > > >>>>>>>>>>>> issues > > >> > > >>>>>>>>>>>>>>> that > > >> > > >>>>>>>>>>>>>>>>>> needs > > >> > > >>>>>>>>>>>>>>>>>>>> to > > >> > > >>>>>>>>>>>>>>>>>>>>> be > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> included in a new hotfix > > >> > > >> release: > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> - Heap-based timers’ restore > > >> > > >>>>>> behaviour > > >> > > >>>>>>>>> is > > >> > > >>>>>>>>>>>>> causing a > > >> > > >>>>>>>>>>>>>>>>>>> critical > > >> > > >>>>>>>>>>>>>>>>>>>>>> recovery > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> issue for StateFun [1] [2] > > >> > > >>> [3]. > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> - There are several > > >> > > >> robustness > > >> > > >>>>>> issues > > >> > > >>>>>>>>> for > > >> > > >>>>>>>>>> the > > >> > > >>>>>>>>>>>>>>> FLIP-27 > > >> > > >>>>>>>>>>>>>>>>> new > > >> > > >>>>>>>>>>>>>>>>>>>>> source > > >> > > >>>>>>>>>>>>>>>>>>>>>> API, > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> such as [4]. We already have > > >> > > >>>> some > > >> > > >>>>>>> users > > >> > > >>>>>>>>>> using > > >> > > >>>>>>>>>>>> the > > >> > > >>>>>>>>>>>>>>>>> FLIP-27 > > >> > > >>>>>>>>>>>>>>>>>>> API > > >> > > >>>>>>>>>>>>>>>>>>>>>> with > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> 1.11.x, > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> so it would be important to > > >> > > >>> get > > >> > > >>>>>> those > > >> > > >>>>>>>>> fixes > > >> > > >>>>>>>>>>> in > > >> > > >>>>>>>>>>>>> for > > >> > > >>>>>>>>>>>>>>>>> 1.11.x > > >> > > >>>>>>>>>>>>>>>>>>> as > > >> > > >>>>>>>>>>>>>>>>>>>>>> well. > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Apart from the issues that are > > >> > > >>>>> already > > >> > > >>>>>>>>> marked > > >> > > >>>>>>>>>> as > > >> > > >>>>>>>>>>>>>>> blocker > > >> > > >>>>>>>>>>>>>>>>> for > > >> > > >>>>>>>>>>>>>>>>>>>>> 1.11.3 > > >> > > >>>>>>>>>>>>>>>>>>>>>> in > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> our > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> JIRA [5], please let us know in > > >> > > >>>> this > > >> > > >>>>>>>>> thread if > > >> > > >>>>>>>>>>>> there > > >> > > >>>>>>>>>>>>>>> is > > >> > > >>>>>>>>>>>>>>>>>>> already > > >> > > >>>>>>>>>>>>>>>>>>>>>> ongoing > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> work for other important fixes > > >> > > >>> that > > >> > > >>>>> we > > >> > > >>>>>>>>> should > > >> > > >>>>>>>>>>> try > > >> > > >>>>>>>>>>>> to > > >> > > >>>>>>>>>>>>>>>>>> include. > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Xintong and I would like to > > >> > > >>>> volunteer > > >> > > >>>>>> for > > >> > > >>>>>>>>>>> managing > > >> > > >>>>>>>>>>>>>>> this > > >> > > >>>>>>>>>>>>>>>>>>> release, > > >> > > >>>>>>>>>>>>>>>>>>>>> and > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> will > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> try to communicate the priority > > >> > > >>> of > > >> > > >>>>>>> pending > > >> > > >>>>>>>>>>>> blockers > > >> > > >>>>>>>>>>>>>>> over > > >> > > >>>>>>>>>>>>>>>>> the > > >> > > >>>>>>>>>>>>>>>>>>>> next > > >> > > >>>>>>>>>>>>>>>>>>>>>> few > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> days. > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Since the aforementioned issues > > >> > > >>> are > > >> > > >>>>>> quite > > >> > > >>>>>>>>>>>> critical, > > >> > > >>>>>>>>>>>>>>> we’d > > >> > > >>>>>>>>>>>>>>>>>> like > > >> > > >>>>>>>>>>>>>>>>>>> to > > >> > > >>>>>>>>>>>>>>>>>>>>> aim > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> for a *feature > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> freeze by the end of next week > > >> > > >>>> (Oct. > > >> > > >>>>>>> 30th)* > > >> > > >>>>>>>>>> and > > >> > > >>>>>>>>>>>>> start > > >> > > >>>>>>>>>>>>>>> the > > >> > > >>>>>>>>>>>>>>>>>>>> release > > >> > > >>>>>>>>>>>>>>>>>>>>>> voting > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> process the week after. > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If that is too short of a > > >> > > >> notice > > >> > > >>>> and > > >> > > >>>>>> you > > >> > > >>>>>>>>> might > > >> > > >>>>>>>>>>>> need > > >> > > >>>>>>>>>>>>>>> more > > >> > > >>>>>>>>>>>>>>>>>> time, > > >> > > >>>>>>>>>>>>>>>>>>>>>> please > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> let > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> us know! > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Cheers, > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Gordon > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> [1] > > >> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-19692 > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> [2] > > >> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-19741 > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> [3] > > >> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-19748 > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> [4] > > >> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-19717 > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> [5] > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>> > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>> > > >> > > >>>>>>>>>> > > >> > > >>>>>>>>> > > >> > > >>>>>>> > > >> > > >>>>>> > > >> > > >>>>> > > >> > > >>>> > > >> > > >>> > > >> > > >> > > >> > > > > >> > > > >> > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20priority%20%3D%20Blocker%20AND%20fixVersion%20%3D%201.11.3 > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>>> -- > > >> > > >>>>>>>>>>>>>>>>>> Best, Jingsong Lee > > >> > > >>>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>>> > > >> > > >>>>>>>>>>>>> > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>> > > >> > > >>>>>>>>>> > > >> > > >>>>>>>>> > > >> > > >>>>>>>> > > >> > > >>>>>>> > > >> > > >>>>>> > > >> > > >>>>> > > >> > > >>>> > > >> > > >>> > > >> > > >> > > >> > > > > >> > > > > >> > > > > >> > > > >> > > > > > >
