Hi, I can work on a PR to backport the relevant changes, but it may take a few days before I get to it.
Thanks, Thomas -- sent from mobile On Mon, Jun 11, 2018, 1:50 PM Tzu-Li (Gordon) Tai <tzuli...@apache.org> wrote: > Thanks for bring this up, Bowen. > > I’m not sure that it’ll be straightforward to just use the Kinesis > connector in Flink 1.5 to work with 1.4. > So, +1 to have a backport of necessary changes to allow the Kinesis > connector to continue to work for Flink 1.4, and we proceed with another > bug fix release for the 1.4.x series. > > We might want to consider only cherry-picking the Kinesis connector > backports on-top of 1.4.2, instead of branching out from 1.4-SNAPSHOT > directly, to minimize the test efforts that would be required for releasing. > But this should probably be discussed on a separate thread of its own. > > Cheers, > Gordon > > > On 11 June 2018 at 12:29:10 PM, Till Rohrmann (trohrm...@apache.org) > wrote: > > I agree that it would be nice to offer a backport for all 1.4 users. > Thomas > would it be possible to contribute the backport back to the community? > > Cheers, > Till > > On Wed, Jun 6, 2018 at 5:25 PM Thomas Weise <t...@apache.org> wrote: > > > --> > > > > On Wed, Jun 6, 2018 at 1:06 AM, Bowen Li <bowenl...@gmail.com> wrote: > > > > > Hi, > > > > > > I think the following email thread might have gone lost. > > > > > > Dyana brought up the attention that AWS has informed users that KPL > > > versions > > > < 0.12.6 will *stop working* starting from the 12th of June. Flink 1.4 > is > > > using KPL 0.12.5 and Flink 1.5 uses 0.12.6, *so Flink 1.4 and below > (1.3, > > > etc) will be impacted.* > > > > > > I think we probably should try our best to communicate this out to > our > > > users in user email alias and all other possible channels, in order > not > > to > > > disrupt their production pipeline. > > > > > > A quick solution we can suggest to users is to package and use > > > flink-connector-kinesis in Flink 1.5. Given that the public APIs > don't > > > change, flink-connector-kinesis in Flink 1.5 should work with Flink > 1.4 > > and > > > below, but that needs verification. > > > > > > > The connector might work, but it is going to require a little extra > > juggling for the users to exclude the 1.5 transitive dependencies. > > > > How about just backporting the changes? We internally use the newer SDK > > with a number of other consumer patches on top of 1.4.2 and it works > fine. > > > > Normally such changes don't qualify for a patch version, but since it > is > > going to stop working anyways, there is probably no harm in this > instance. > > > > Thomas > > > > > > > > > What do you think? > > > > > > Thanks, Bowen > > > > > > > > > ---------- Forwarded message ---------- > > > From: Bowen Li <bowenl...@gmail.com> > > > Date: Fri, May 11, 2018 at 10:28 AM > > > Subject: Re: KPL in current stable 1.4.2 and below, upcoming problem > > > To: dev@flink.apache.org, "Tzu-Li (Gordon) Tai" <tzuli...@apache.org> > > > > > > > > > > Thanks, this is a great heads-up! Flink 1.4 is using KPL 0.12.5, *so > > Flink > > > version 1.4 or below will be effected.* > > > > > > Kinesis sink is in flink-kinesis-connector. Good news is that whoever > is > > > using flink-kinesis-connector right now is building it themself, > because > > > Flink doesn't publish that jar into maven due to licensing issue. So > > these > > > users, like you Dyana, already have build experience and may try to > bump > > > KPL themselves. > > > > > > I think it'll be great if Flink can bump KPL in Flink 1.2/1.3/1.4 and > > > release minor versions for them, as an official support. It also > requires > > > checking backward compatibility. This can be done after releasing > 1.5. > > > @Gordon may take the final call of how to eventually do it. > > > > > > Thanks, > > > Bowen > > > > > > > > > On Thu, May 10, 2018 at 1:35 AM, Dyana Rose <dyana.r...@salecycle.com> > > > > wrote: > > > > > > > Hello, > > > > > > > > We've received notification from AWS that the Kinesis Producer > Library > > > > versions < 0.12.6 will stop working after the 12th of June > (assuming > > the > > > > date in the email is in US format...) > > > > > > > > Flink v1.5.0 has the KPL version at 0.12.6 so it will be fine when > it's > > > > released. However using the kinesis connector in any previous > version > > > looks > > > > like they'll have an issue. > > > > > > > > I'm not sure how/if you want to communicate this. We build Flink > > > ourselves, > > > > so I plan on having a look at any changes done to the Kinesis Sink > in > > > > v1.5.0 and then bumpimg the KPL version in our fork and rebuilding. > > > > > > > > Thanks, > > > > Dyana > > > > > > > > below is the email we received (note: we're in eu-west-1): > > > > -------- > > > > > > > > Hello, > > > > > > > > > > > > > > > > Your action is required: please update clients running Kinesis > Producer > > > > Library 0.12.5 or older or you will experience a breaking change to > > your > > > > application. > > > > > > > > > > > > > > > > We've discovered you have one or more clients writing data to > Amazon > > > > Kinesis Data Streams running an outdated version of the Kinesis > > Producer > > > > Library. On 6/12 these clients will be impacted if they are not > updated > > > to > > > > Kinesis Producer Library version 0.12.6 or newer. On 06/12 Kinesis > Data > > > > Streams will install ATS certificates which will prevent these > outdated > > > > clients from writing to a Kinesis Data Stream. The result of this > > change > > > > will break any producer using KPL 0.12.5 or older. > > > > > > > > > > > > * How do I update clients and applications to use the latest version > of > > > the > > > > Kinesis Producer Library? > > > > > > > > You will need to ensure producers leveraging the Kinesis Producer > > Library > > > > have upgraded to version 0.12.6 or newer. If you operate older > versions > > > > your application will break due untrusted SSL certification. > > > > > > > > Via Maven install Kinesis Producer Library version 0.12.6 or higher > [2] > > > > > > > > After you've configured your clients to use the new version, you're > > done. > > > > > > > > * What if I have questions or issues? > > > > > > > > If you have questions or issues, please contact your AWS Technical > > > Account > > > > Manager or AWS support and file a support ticket [3]. > > > > > > > > [1] https://docs.aws.amazon.com/streams/latest/dev/kinesis-kpl-u > > > > pgrades.html > > > > > > > > [2] http://search.maven.org/#artifactdetails|com.amazonaws|amazo > > > > n-kinesis-produ... > > > > <http://search.maven.org/#artifactdetails%7Ccom.amazonaws%7C > > > > amazon-kinesis-producer%7C0.12.6%7Cjar> > > > > > > > > [3] https://aws.amazon.com/support > > > > > > > > > > > > > > > > - Amazon Kinesis Data Streams Team > > > > ----- > > > > > > > > > >
