Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

so needs to happen there) > > > > > > e. It's up to the maintainers for DynamoDB and Opensearch to > decide > > > > this > > > > > > themselves, given that I believe they've only been introduced > recently > > > > > and > > > > >

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

> > asap. > > > > > > > > > > Best regards, > > > > > > > > > > Martijn > > > > > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-25726 > > > > > > > > > > > > > &g

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

; > > > > transactions, there is no consensus on solving that problem in > > SinkV2. > > > > Will > > > > > it require any breaking change in SinkV2? > > > > > > > > > > Just want to reiterate my earlier question. Wh

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

> > > deprecating SinkFunction at this point. Dong Lin has raised the > > blocker > > > > > issues of migration multiple times in this thread which I think we > > > should > > > > > address first. I don't know why we rush to deprecate Si

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

bility > > > of > > > > Flink connector APIs. > > > > > > > > We are hoping any issues with the APIs for Iceberg connector will > > surface > > > > > sooner and get more attention from the Flink community when the > > > conne

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

rather than in Iceberg repo. > > > > > > > > > The connector externalizing is a big step for building a mechanism to > > > guarantee Flink connector API is stable and workable. The follow-up > step > > > should be trying the new APIs in externalized connectors and g

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

t; > Iceberg community raised concerns[1] about the workability and > > stability > > > of > > > > Flink connector APIs. > > > > > > > > We are hoping any issues with the APIs for Iceberg connector will > > surface > > > > > sooner

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

to > > > guarantee Flink connector API is stable and workable. The follow-up > step > > > should be trying the new APIs in externalized connectors and giving > users > > > the confidence to migrate. > > > > > > > > > Best, > > >

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

migrate. > > > > > > Best, > > Jark > > > > [1]: https://lists.apache.org/thread/r5zqnkt01x2c611brkjmxxnt3bfcgl1b > > > > On Tue, 7 Feb 2023 at 09:53, yuxia wrote: > > > > > Hi Konstantin, > > > Just FYI, the FileSy

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

xia > > > > ----- 原始邮件 - > > 发件人: "Dong Lin" > > 收件人: "dev" > > 抄送: "Jing Ge" , "Yun Tang" > > 发送时间: 星期二, 2023年 2 月 07日 上午 9:41:07 > > 主题: Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction &

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

> > Best regards, > Yuxia > > - 原始邮件 - > 发件人: "Dong Lin" > 收件人: "dev" > 抄送: "Jing Ge" , "Yun Tang" > 发送时间: 星期二, 2023年 2 月 07日 上午 9:41:07 > 主题: Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction > > Hi

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

Hi Konstantin, Just FYI, the FileSystemTableSink are still using SinkFunction. Best regards, Yuxia - 原始邮件 - 发件人: "Dong Lin" 收件人: "dev" 抄送: "Jing Ge" , "Yun Tang" 发送时间: 星期二, 2023年 2 月 07日 上午 9:41:07 主题: Re: [DISCUSS] Promote SinkV2 to @Public a

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

Hi Konstantin, Thanks for the reply. Please see my comments inline. On Mon, Feb 6, 2023 at 9:48 PM Konstantin Knauf wrote: > Hi Steven, > > Sink is already deprecated. It was deprecated at the moment where we > introduced SinkV2. > > Hi Jark, Hi Dong, > > My understanding is the SinkV2 is a wor

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

Hi, We have the same goal to let users migrate to SinkV2. Just marking those interfaces as deprecated or public won't achieve this goal. Frankly speaking, users will start to migrate not because those interfaces are marked as deprecated. They will do it because there is a workable implementation w

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

Recently, a critical bug with the Unified Sink committer was reported: [FLINK-30238] Unified Sink committer does not clean up state on final savepoint - ASF JIRA (apache.org) . Fabian Paul reported: Hi folks, > > I did some initial investigation

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

Hi Steven, Sink is already deprecated. It was deprecated at the moment where we introduced SinkV2. Hi Jark, Hi Dong, My understanding is the SinkV2 is a workable interface. The most important connectors have been migrated (Kafka, Filesystem) and more connectors (OpenSearch, ElasticSearch, Kinesi

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

I agree with Dong Lin. Oracle explains how to use Deprecate API [1]: You are strongly recommended to use the Javadoc @deprecated tag with > appropriate comments explaining how to use the new API. This ensures > developers will *have a workable migration path from the old API to the > new API*.

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

Regarding the discussion on global committer [1] for sinks with global transactions, there is no consensus on solving that problem in SinkV2. Will it require any breaking change in SinkV2? Also will SinkV1 be deprecated too? or it should happen sometime after SinkFunction deprecation? [1] https:/

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

Hi Konstantin, Thanks for the comment! Please see my comment inline. Cheers, Dong On Sat, Feb 4, 2023 at 2:06 AM Konstantin Knauf wrote: > Hi everyone, > > sorry for joining the discussion late. > > 1) Is there an option to deprecate SinkFunction in Flink 1.17 while leaving > SinkV2 @PublicEvo

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

Hi everyone, sorry for joining the discussion late. 1) Is there an option to deprecate SinkFunction in Flink 1.17 while leaving SinkV2 @PublicEvolving in Flink 1.17. We then aim to make SinkV2 @Public in and remove SinkFunction in Flink 1.18. @PublicEvolving are intended for public use. So, I don

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

Hi Martijn, Thanks for driving this effort to clean-up the Flink codebase! I like the idea to cleanup Flink codebase to avoid having two Sinks. On the other hand, I also thing the concern mentioned by Jing makes sense. In addition to thinking in terms of the rule proposed in FLIP-197

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

, Yuxia - 原始邮件 - 发件人: "Jing Ge" 收件人: "dev" 抄送: "Yun Gao" , "Qingsheng Ren" , "Jark Wu" 发送时间: 星期二, 2023年 1 月 24日 下午 7:25:59 主题: Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction Hi Martijn, Thanks for sharing more info

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

Hi Martijn, Thanks for sharing more information to help us have a clear big picture. 1) I agree with you. We should graduate the SinkV2 API asap. I have also tried many times in the past and there were always some issues that postponed the graduation and they made sense. One rule to check if an A

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

Hi Jing, Thanks for your input! 1) I think that we have had more than two release cycles for the Sink V2 API. The first release of the Sink API was introduced with Flink 1.12 which was December 2020, more than 2 years ago. The additional feature that Sink V2 has introduced was the ability to hook

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

Hi, There are some typos in my last email, although hopefully I made myself clear with the context, sorry about that: FLIP-179 -> FLIP-197 FlinkKafkaConsumer -> FlinkKafkaProducer Best regards, Jing On Thu, Jan 19, 2023 at 5:30 AM Jing Ge wrote: > Hi, > > Thanks Martijn for bringing this to o

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

Hi, Thanks Martijn for bringing this to our attention. This discussion deserves a much wider range of attention. because graduation of the Sink API will have an impact on all connectors. For me, briefly speaking, like I already pointed out in another thread [1], it is too hasty and risky to do it

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

Hi Martijn, Thanks for bringing us this discussion! I think it's time to mark SinkFunction as deprecated. It may help a lot to encourage users to migrate existing sink connectors to the new interface. About the concern of Lijie, I'm not sure if it's OK to compatibly change the interface with @Pu

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

Hi Martijn, Thanks for driving this. I have a only concern about the Sink.InitContext. Does the Sink.InitContext will also be changed to @Public ? As described in FLIP-287, currently the Sink.InitContext still lacks some necessary information to migrate existing connectors to new sinks. If it is

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

SinkV2 was introduced in Flink-1.15 and annotated as @PublicEvolving from the 1st day [1]. From FLIP-197, we can promote it to @Public since it already existed with two releases. And I didn't find a FLIP to discuss the process to deprecate APIs, considering the SinkFunction has actually been sta