Stephan, it will be great if we can also backport the DataStreamUtils
related commits that help with collecting output from unbounded streams.
e.g.
https://github.com/apache/flink/commit/09a7a66b7313fea64817fe960a8da1265b428efc
I tried to copy and paste the code to unblock myself. but it quickly got
into the rabbit hole of more and more code.
On Fri, Oct 30, 2020 at 11:02 AM Stephan Ewen wrote:
> I have started with backporting the source API changes. Some minor
> conflicts to solve, will need a bit more to finish this.
>
> On Fri, Oct 30, 2020 at 7:25 AM Tzu-Li (Gordon) Tai
> wrote:
>
> > @Stephan Ewen
> > Are there already plans or ongoing efforts for backporting the list of
> > FLIP-27 changes that you posted?
> >
> > On Thu, Oct 29, 2020 at 7:08 PM Xintong Song
> > wrote:
> >
> >> Hi folks,
> >>
> >> Just to provide some updates concerning the status on the
> >> test instabilities.
> >>
> >> Currently, we have 30 unresolved tickets labeled with `Affects Version`
> >> 1.11.x.
> >>
> >>
> https://issues.apache.org/jira/browse/FLINK-19775?filter=12348580&jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20affectedVersion%20in%20(1.11.0%2C%201.11.1%2C%201.11.2%2C%201.11.3)%20AND%20labels%20%3D%20test-stability%20ORDER%20BY%20created%20DESC
> >>
> >> Among the 30 tickets, 11 of them are:
> >> - Have occured in the recent 3 months
> >> - Not confirmed to be pure testability issues
> >> - Not confirmed to be rare condition cases
> >>
> >> It would be nice if someone familiar with these components can take a
> look
> >> into these issues.
> >>
> >> - https://issues.apache.org/jira/browse/FLINK-17159 (ES6)
> >> - https://issues.apache.org/jira/browse/FLINK-17912 (Kafka)
> >> - https://issues.apache.org/jira/browse/FLINK-17949 (Kafka)
> >> ⁃ https://issues.apache.org/jira/browse/FLINK-18444 (Kafka)
> >> - https://issues.apache.org/jira/browse/FLINK-18634 (Kafka)
> >> - https://issues.apache.org/jira/browse/FLINK-18648 (Kafka)
> >> - https://issues.apache.org/jira/browse/FLINK-18807 (Kafka)
> >> - https://issues.apache.org/jira/browse/FLINK-19369 (BlobClientTest)
> >> - https://issues.apache.org/jira/browse/FLINK-19436 (TPCDS)
> >> - https://issues.apache.org/jira/browse/FLINK-19690 (Format/Parquet)
> >> - https://issues.apache.org/jira/browse/FLINK-19775
> >> (SystemProcessingTimeServiceTest)
> >>
> >> Thank you~
> >>
> >> Xintong Song
> >>
> >>
> >>
> >> On Thu, Oct 29, 2020 at 10:21 AM Jingsong Li
> >> wrote:
> >>
> >> > +1 to backport the FLIP-27 adjustments to 1.11.x.
> >> >
> >> > If possible, that would be great. Many people are looking forward to
> the
> >> > FLIP-27 interface, but they don't want to take the risk to upgrade to
> >> 1.12
> >> > (And wait 1.12). After all, 1.11 is a relatively stable version.
> >> >
> >> > Best,
> >> > Jingsong
> >> >
> >> > On Thu, Oct 29, 2020 at 1:24 AM Stephan Ewen
> wrote:
> >> >
> >> > > Thanks for starting this.
> >> > >
> >> > > +1 form my side to backport the FLIP-27 adjustments to 1.11.x.
> >> > >
> >> > > There were quite a few changes, and I think we need to cherry-pick
> >> them
> >> > all
> >> > > to not get some inconsistent mix of changes and many merge
> conflicts.
> >> > > I made a list below of what we need to add to "release-1.11".
> >> > >
> >> > > * Core Source API Changes to backport (in REVERSE order)*
> >> > >
> >> > > (Use: "git log
> >> > > flink-core/src/main/java/org/apache/flink/api/connector/source")
> >> > >
> >> > > commit 162c072e9265a7b6dd9d6f5459eb7974091c4c4e
> >> > > [FLINK-19492][core] Consolidate Source Events between Source API and
> >> > Split
> >> > > Reader API
> >> > >
> >> > > commit 428fbbad85379f71a9fbf1a1ab3e3dcfa9006436
> >> > > [FLINK-19205][core] Add access to configuration and hostname in the
> >> > > SourceReaderContext
> >> > >
> >> > > commit 723e1790744ddc78d19c7c978442af1383f38d33
> >> > > [hotfix][core] Add to Source Enumerator convenience methods to
> assign
> >> > > single split
> >> > >
> >> > > * Connector Base Changes to Backport (in REVERSE order)*
> >> > >
> >> > > (Use: "git log flink-connectors/flink-connector-base")
> >> > >
> >> > > commit 401f56fe9d6b0271260edf9787cdcbfe4d03874d
> >> > > [FLINK-19427][FLINK-19489][tests] Fix test conditions for
> >> > > 'SplitFetcherTest.testNotifiesWhenGoingIdleConcurrent()'
> >> > >
> >> > > commit 68c5c2ff779d82a1ff81ffaf60d8a1b283797db1
> >> > > [FLINK-19448][connector base] Explicitly check for un-expected
> >> condition
> >> > > that would leave an inconsistent state
> >> > >
> >> > > commit 162c072e9265a7b6dd9d6f5459eb7974091c4c4e
> >> > > [FLINK-19492][core] Consolidate Source Events between Source API and
> >> > Split
> >> > > Reader API
> >> > >
> >> > > commit c1ca7a4c7c21ec8868c14c43c559625b794c
> >> > > [refactor][tests] Move some source test utils from
> >> flink-connector-base
> >> > to
> >> > > flink-core
> >> > >
> >> > > commit ee5c4c211c35c70d28252363bbc8400453609977
> >> > > [FLINK-19251][connectors] Avoi
