[RESULT][VOTE] FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-14 Thread Dawid Wysakowicz
The vote for FLIP-510: Drop ChangelogNormalize for operations which don't need it [1] has concluded (discussion thread [2]). The vote will be closed [3]. There were 8 approving votes, 7 binding, and there were no disapprovals: - Timo Walther (binding) - Leonard Xu (binding) - X

[jira] [Created] (FLINK-37474) FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-14 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-37474: Summary: FLIP-510: Drop ChangelogNormalize for operations which don't need it Key: FLINK-37474 URL: https://issues.apache.org/jira/browse/FLINK-37474 Pr

Re: [VOTE] FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-14 Thread Dawid Wysakowicz
; > > >> > > > >> -- > > > >> > > > >> Best! > > > >> Xuyang > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> 在 2025-0

Re: [VOTE] FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-13 Thread Jiabao Sun
binding) > > >> > > > >> >Best, > > >> >Leonard > > >> > > > >> >> 2025年3月11日 18:42,Timo Walther 写道: > > >> >> > > >> >> +1 (binding) > > >> >> > > >> &

Re: [VOTE] FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-12 Thread Martijn Visser
19:12:05,"Leonard Xu" 写道: > >> >+1(binding) > >> > > >> >Best, > >> >Leonard > >> > > >> >> 2025年3月11日 18:42,Timo Walther 写道: > >> >> > >> >> +1 (binding) > >> >> > >> >> T

Re: [VOTE] FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-12 Thread Yuepeng Pan
>> >> >> Thanks for working on this! >> >> >> >> Timo >> >> >> >> On 11.03.25 11:05, Dawid Wysakowicz wrote: >> >>> Hi everyone, >> >>> I'd like to start a vote on FLIP-510: Drop ChangelogNormalize

Re: Re: [VOTE] FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-12 Thread Sergey Nuyanzin
025年3月11日 18:42,Timo Walther 写道: > >> > >> +1 (binding) > >> > >> Thanks for working on this! > >> > >> Timo > >> > >> On 11.03.25 11:05, Dawid Wysakowicz wrote: > >>> Hi everyone, > >>> I'd like to

Re: FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-11 Thread Timo Walther
Hi Dawid, thanks for this proposal. This is a very nice improvement to the SQL engine. Changelog normalize is a very state-intensive operation. Any possibility to avoid it should be implemented. Updating ChangelogMode is an elegant solution and avoid updating the RowKind. When we introduced

[VOTE] FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-11 Thread Dawid Wysakowicz
Hi everyone, I'd like to start a vote on FLIP-510: Drop ChangelogNormalize for operations which don't need it[1] which has been discussed in this thread [2]. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1] https://cwiki.apache.org/co

Re:Re: [VOTE] FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-11 Thread Xuyang
; >> On 11.03.25 11:05, Dawid Wysakowicz wrote: >>> Hi everyone, >>> I'd like to start a vote on FLIP-510: Drop ChangelogNormalize for >>> operations which don't need it[1] >>> which has been discussed in this thread [2]. >>> The vote

Re: [VOTE] FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-11 Thread Timo Walther
+1 (binding) Thanks for working on this! Timo On 11.03.25 11:05, Dawid Wysakowicz wrote: Hi everyone, I'd like to start a vote on FLIP-510: Drop ChangelogNormalize for operations which don't need it[1] which has been discussed in this thread [2]. The vote will be open for at leas

Re: [VOTE] FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-11 Thread Leonard Xu
+1(binding) Best, Leonard > 2025年3月11日 18:42,Timo Walther 写道: > > +1 (binding) > > Thanks for working on this! > > Timo > > On 11.03.25 11:05, Dawid Wysakowicz wrote: >> Hi everyone, >> I'd like to start a vote on FLIP-510: Drop ChangelogNorma

Re: FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-11 Thread Dawid Wysakowicz
Thank you for the feedback. I updated the name in the Flip and I'll start the vote in a separate thread. On Tue, 11 Mar 2025 at 09:21, Leonard Xu wrote: > +1 for this proposal after went through Dawid and Xuyang’s discussion, and > I think we can start a vote if there’re no objections. > > For t

Re: FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-11 Thread Leonard Xu
+1 for this proposal after went through Dawid and Xuyang’s discussion, and I think we can start a vote if there’re no objections. For the naming, shorter is better in this case, +1 for ChangelogMode.keyOnlyDeletes() after discussed with ChatGPT. Best, Leonard

Re:Re: Re: Re: FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-09 Thread Xuyang
I have no other questions. +1 for it. -- Best! Xuyang At 2025-03-07 19:37:09, "Dawid Wysakowicz" wrote: >> >> From my understanding, for a sink, if its schema includes a primary key, >> we can assume it has >> the ability to process delete messages (with '-D') and perform deletio

Re: Re: Re: FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-07 Thread Dawid Wysakowicz
> > From my understanding, for a sink, if its schema includes a primary key, > we can assume it has > the ability to process delete messages (with '-D') and perform deletions > by key (PK). If it does not > include a PK, we would implicitly treat it as a log-structured table that > supports full ro

Re:Re: Re: FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-02 Thread Xuyang
Hi, Dawid. Thanks for your response. I believe I've identified a key point, but I’m a bit unclear about the following you said. Could you please provide an example for clarification? ``` The only missing information is if the external sink can consume deletes by key and if a source produces

Re: Re: FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-02-28 Thread Dawid Wysakowicz
Hey Xuyang, Ad. 1 Yes, you're right, but we already do that for determining if we need UPDATE_BEFORE or not. FlinkChangelogModeInferenceProgram already deals with that. Ad. 2 Unfortunately it is. This is also the only reason I need a FLIP. We can determine internally for every internal operator if

Re:Re: FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-02-27 Thread Xuyang
Hi Dawid. Big +1 for this FLIP. After reading through it, I have a few questions and would appreciate your responses: 1. IIUC, we only need to provide additional information in the `FlinkChangelogModeInferenceProgram` to enable the inference program to determine whether it is safe to remov

Re: FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-02-27 Thread Martijn Visser
Hi Dawid, Thanks for the FLIP, looks like a good improvement for me that will bring a lot of benefits. +1 Best regards, Martijn On Tue, Feb 25, 2025 at 6:51 AM Sergey Nuyanzin wrote: > +1 for such improvement > > On Mon, Feb 24, 2025 at 12:01 PM Dawid Wysakowicz > wrote: > > > > Hi everyone,

Re: FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-02-25 Thread Sergey Nuyanzin
+1 for such improvement On Mon, Feb 24, 2025 at 12:01 PM Dawid Wysakowicz wrote: > > Hi everyone, > > I would like to initiate a discussion for the FLIP-510[1] below, which aims > on optimising certain use cases in SQL which at the moment add > ChangelogNormalize, but don't necessarily need to do

FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-02-24 Thread Dawid Wysakowicz
Hi everyone, I would like to initiate a discussion for the FLIP-510[1] below, which aims on optimising certain use cases in SQL which at the moment add ChangelogNormalize, but don't necessarily need to do it. Looking forward to hearing from you. [1] https://cwiki.apache.org/confluence/x/7o5EF