Hi everyone, It seems we are all aligned that this is a useful feature - thanks for the feedback. I think we can now move on to the vote.
Kind regards, Gustavo On Tue, 10 Nov 2026 at 17:01, Gustavo de Morais <[email protected]> wrote: > Hi Ferenc and Hao, > > Thanks for the feedback, I'm glad you find this useful. > > Ferenc, our discussion was one of the motivating factors, and I also > mentioned it in the FLIP - thanks for bringing it up. With TO_APPEND, we > cover this common pattern with one easy PTF and keep each function focused > on a single task: TO_CHANGELOG turns a table into a changelog with its > different event types (inserts, updates and deletes), while TO_APPEND > simply converts an updating table into an append-only table, without > exposing the changelog concept. > > Kind regards, > Gustavo > > On Mon, 9 Nov 2026 at 20:26, Hao Li <[email protected]> wrote: > >> Hi Gustavo, >> >> Thanks for the proposal. This is a very convenient function I've been >> hoping for. +1 from my side. >> >> Hao >> >> On Mon, Nov 9, 2026 at 10:15 AM Ferenc Csaky via dev >> <[email protected]> wrote: >> > >> > Hi Gustavo, >> > >> > Thanks for driving this! +1 form my side, this is pretty much what I >> already do >> > downstream instead of using TO_CHANGELOG when i don't care about the >> input OP. >> > >> > I should have been suggested this direction myself as well on that >> discussion as >> > an option. :) >> > >> > Anyways, I am glad to see this will be part of the framework. >> > >> > Best, >> > Ferenc >> > >> > On Mon, Nov 9, 2026, at 18:58, Gustavo de Morais wrote: >> > > Hi Timo, >> > > >> > > Thanks a lot for the feedback and input. I've added your suggestion to >> > > Future Work. >> > > >> > > Kind regards, >> > > Gustavo >> > > >> > > On Mon, 9 Nov 2026 at 10:28, Timo Walther via dev < >> [email protected]> >> > > wrote: >> > > >> > > > Hi Gustavo, >> > > > >> > > > Thanks for proposing this FLIP. Big +1 for this effort. What others >> > > > might not see is that we are still in the learning process of how we >> > > > want to use the PTF framework for our built-in functions. >> Overloading >> > > > single functions with too much functionality is definitely an issue >> that >> > > > happened with TO/FROM_CHANGELOG. A TO_APPEND would most likely have >> > > > solved 80% of all use cases, and I agree that we should make the >> most >> > > > common case as convenient as possible. >> > > > >> > > > As an optimization, we might even be able to remove the >> > > > ChangelogNormalize if a TO_APPEND immediately follows it. Maybe we >> can >> > > > add this to the Future Work section. >> > > > >> > > > Thanks, >> > > > Timo >> > > > >> > > > >> > > > On 08.11.26 07:20, Gustavo de Morais wrote: >> > > > > Hi everyone, >> > > > > >> > > > > I'd like to propose FLIP-621: Add built-in TO_APPEND PTF [1] for >> > > > > discussion. Since we added TO_CHANGELOG, we've seen the same >> pattern come >> > > > > up frequently: users have an updating table and want to write its >> inserts >> > > > > and updates to an append-only sink, dropping deletes. >> TO_CHANGELOG can do >> > > > > this, but it needs an op_mapping that drops deletes plus a >> projection >> > > > that >> > > > > removes the op column. >> > > > > >> > > > > TO_APPEND gives everyone else a very easy way to stay in the >> append >> > > > world, >> > > > > without having to understand the changelog behind it. Full >> signature: >> > > > > >> > > > > SELECT * FROM TO_APPEND(input => TABLE my_table); >> > > > > >> > > > > Inserts and updates are emitted as inserts, update-befores and >> deletes >> > > > are >> > > > > dropped, and the input schema stays unchanged. Following the PTF >> > > > guidelines >> > > > > [2], we want to provide functions that solve one problem at a >> time. >> > > > > TO_APPEND fits that well: it is a stateless, row-semantics PTF >> with an >> > > > easy >> > > > > to reason implementation. It also covers the use case discussed >> in the >> > > > > mailing list for adding one more parameter to TO_CHANGELOG [3]. >> Looking >> > > > > forward to your feedback and thoughts. >> > > > > >> > > > > Kind regards, >> > > > > Gustavo >> > > > > >> > > > > [1] >> > > > > >> > > > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-621%3A+Add+built-in+TO_APPEND+PTF >> > > > > [2] >> > > > > >> > > > >> https://cwiki.apache.org/confluence/spaces/FLINK/pages/462816208/Built-in+Process+Table+Function+Guidelines >> > > > > [3] >> https://lists.apache.org/thread/pfd6ktpyh95j126jm0qolngkrm2bpyhg >> > > > >> > > > >> > > >> >
