Hi all, I want to give an update from my perspective.
flink-connector-kinesis' two critical dependencies KPL and Dynamo stream adapter haven't been converted to Apache 2.0 license yet. The license-changing PRs [1] are still pending, for ~2 months so far. I've left comments there, and hope they can be merged and have new releases soon. Given that, as well as some major work required to adapt flink-connector-kinesis code to new AWS libs (described by Dyana in the ticket), I personally don't think we can make it in 1.9. I would recommend re-aiming it at 1.10. [1] KPL https://github.com/awslabs/amazon-kinesis-producer/pull/256 Dynamo stream adapter https://github.com/awslabs/dynamodb-streams-kinesis-adapter/pull/23 On Tue, Jun 18, 2019 at 6:09 AM Dyana Rose <dyana.r...@salecycle.com> wrote: > I've pushed an early WIP of the code changes to our fork and added a long > comment on the work that's been done, what issues I've come across, and > requests for discussion on those issues > > https://issues.apache.org/jira/browse/FLINK-12847 > > Thanks, > Dyana > > On Fri, 14 Jun 2019 at 10:34, Aljoscha Krettek <aljos...@apache.org> > wrote: > > > +1 > > > > Nice! Less special-case handling is always good. > > > > > On 14. Jun 2019, at 10:30, Thomas Weise <t...@apache.org> wrote: > > > > > > Dyana, thanks for taking this up! > > > > > > The flink-connector-kinesis module is already part of the CI pipeline, > it > > > is just excluded when creating the release. So what needs to be done is > > to > > > remove the -Pinclude-kinesis cruft and make it part of the default > > modules > > > instead. > > > > > > Thomas > > > > > > > > > On Fri, Jun 14, 2019 at 10:06 AM Dyana Rose <dyana.r...@salecycle.com> > > > wrote: > > > > > >> Brilliant. That Issue is in now under: > > >> https://issues.apache.org/jira/browse/FLINK-12847 > > >> > > >> Thanks, > > >> Dyana > > >> > > >> On Fri, 14 Jun 2019 at 03:07, Tzu-Li (Gordon) Tai < > tzuli...@apache.org> > > >> wrote: > > >> > > >>> Hi, > > >>> > > >>> Thanks Dyana for bringing this up and Bowen for helping to move this > > >>> forward. Very happy to hear about this! > > >>> > > >>> Please feel free to to create a new JIRA ticket for this and assign > it > > to > > >>> yourself. +1 to aim this for 1.9.0. > > >>> The subtasks that Bowen mentioned looks good to me. Issues such as > [1] > > >> and > > >>> [2] should also be resolved as part of this effort. > > >>> > > >>> Also cc'ing Thomas, who recently has more coverage on the Flink > Kinesis > > >>> connector. > > >>> > > >>> Cheers, > > >>> Gordon > > >>> > > >>> [1] https://issues.apache.org/jira/browse/FLINK-3924 > > >>> [2] https://issues.apache.org/jira/browse/FLINK-7673 > > >>> > > >>> On Fri, Jun 14, 2019 at 2:46 AM Bowen Li <bowenl...@gmail.com> > wrote: > > >>> > > >>>> Hi Dyana, > > >>>> > > >>>> Thanks for bringing this up! > > >>>> > > >>>> You are right that ASL is the blocker for us to officially include > > >>>> flink-connecotr-kinesis as a connector module to build and publish > to > > >>> Maven > > >>>> central. I've been thru the mess of building, publishing, and > > >> maintaining > > >>>> flink-connector-kinesis via Jfrog, and that's a really really > painful > > >>>> experience... Glad to hear AWS finally pulls the trigger to change > > >>>> KCL/KPL's license. So big +1 on this initiative from me. > > >>>> > > >>>> I'm not aware of any previous discussion on this, so please feel > free > > >> to > > >>>> create a new JIRA ticket, assign to yourself, and work on it. As a > > >>>> committer, I'll be happy to help move this effort forward, and we > can > > >>> seek > > >>>> help from other experts in kinesis connector like @Tzu-Li (Gordon) > Tai > > >>>> <tzuli...@apache.org> when needed. > > >>>> > > >>>> The task should include, but not limited to, upgrading KCL/KPL to > new > > >>>> versions of Apache 2.0 license, changing licenses and NOTICE files > in > > >>>> flink-connector-kinesis, and adding flink-connector-kinesis to > build, > > >> CI > > >>>> and artifact publishing pipeline. These can be broken into subtasks. > > >>>> > > >>>> If AWS's PR you gave can be finished soon enough, we may be able to > > >> sneak > > >>>> this into Flink 1.9 before feature freeze which is currently set as > > end > > >>> of > > >>>> June. Otherwise, we may have to wait till the next major release > like > > >>> 1.10, > > >>>> as such a big change may not happen in maintenance releases like > > 1.9.1. > > >>>> > > >>>> Bowen > > >>>> > > >>>> On Thu, Jun 13, 2019 at 5:38 AM dyana.rose < > dyana.r...@salecycle.com> > > >>>> wrote: > > >>>> > > >>>>> > > >>>>> The Kinesis Client Library v2.x and the AWS Java SDK v2.x both are > > now > > >>> on > > >>>>> the Apache 2.0 license. > > >>>>> > > >>> > > https://github.com/awslabs/amazon-kinesis-client/blob/master/LICENSE.txt > > >>>>> https://github.com/aws/aws-sdk-java-v2/blob/master/LICENSE.txt > > >>>>> > > >>>>> There is a PR for the Kinesis Producer Library to update it to the > > >>> Apache > > >>>>> 2.0 license ( > > >>> https://github.com/awslabs/amazon-kinesis-producer/pull/256) > > >>>>> > > >>>>> If I understand the Amazon software license issue correctly > updating > > >> to > > >>>>> these new major versions (and the KPL when it's available under the > > >>> Apache > > >>>>> license) will allow the Kinesis connectors to be distributed in the > > >> core > > >>>>> build. (making my life easier) > > >>>>> > > >>>>> I haven't seen a Jira ticket specifically for an upgrade in major > > >>>>> version, but it would solve this one, though otherwise than > intended! > > >>>>> https://issues.apache.org/jira/browse/FLINK-7673 > > >>>>> > > >>>>> Unless there are already discussed reasons not to upgrade, I'll > stick > > >> a > > >>>>> ticket in for it and cross my fingers that the KPL PR gets merged > > >>> sometime > > >>>>> in the relatively near future. > > >>>>> > > >>>>> Thanks, > > >>>>> Dyana > > >>>>> > > >>>> > > >>> > > >> > > >> > > >> -- > > >> > > >> Dyana Rose > > >> Software Engineer > > >> > > >> > > >> W: www.salecycle.com <http://www.salecycle.com/> > > >> [image: The 2019 Look Book - Download Now] > > >> <https://t.xink.io/Tracking/Index/WcwBAKNtAAAwphkA0> > > >> > > > > > > -- > > Dyana Rose > Software Engineer > > > W: www.salecycle.com <http://www.salecycle.com/> > [image: Airline & Travel Booking Trends - Download Report] > <https://t.xink.io/Tracking/Index/vM4BAKNtAAAwphkA0> >
