Hi everyone,

It seems we are all aligned that this is a useful feature - thanks for the
feedback. I think we can now move on to the vote.

Kind regards,
Gustavo

On Tue, 10 Nov 2026 at 17:01, Gustavo de Morais <[email protected]>
wrote:

> Hi Ferenc and Hao,
>
> Thanks for the feedback, I'm glad you find this useful.
>
> Ferenc, our discussion was one of the motivating factors, and I also
> mentioned it in the FLIP - thanks for bringing it up. With TO_APPEND, we
> cover this common pattern with one easy PTF and keep each function focused
> on a single task: TO_CHANGELOG turns a table into a changelog with its
> different event types (inserts, updates and deletes), while TO_APPEND
> simply converts an updating table into an append-only table, without
> exposing the changelog concept.
>
> Kind regards,
> Gustavo
>
> On Mon, 9 Nov 2026 at 20:26, Hao Li <[email protected]> wrote:
>
>> Hi Gustavo,
>>
>> Thanks for the proposal. This is a very convenient function I've been
>> hoping for. +1 from my side.
>>
>> Hao
>>
>> On Mon, Nov 9, 2026 at 10:15 AM Ferenc Csaky via dev
>> <[email protected]> wrote:
>> >
>> > Hi Gustavo,
>> >
>> > Thanks for driving this! +1 form my side, this is pretty much what I
>> already do
>> > downstream instead of using TO_CHANGELOG when i don't care about the
>> input OP.
>> >
>> > I should have been suggested this direction myself as well on that
>> discussion as
>> > an option. :)
>> >
>> > Anyways, I am glad to see this will be part of the framework.
>> >
>> > Best,
>> > Ferenc
>> >
>> > On Mon, Nov 9, 2026, at 18:58, Gustavo de Morais wrote:
>> > > Hi Timo,
>> > >
>> > > Thanks a lot for the feedback and input. I've added your suggestion to
>> > > Future Work.
>> > >
>> > > Kind regards,
>> > > Gustavo
>> > >
>> > > On Mon, 9 Nov 2026 at 10:28, Timo Walther via dev <
>> [email protected]>
>> > > wrote:
>> > >
>> > > > Hi Gustavo,
>> > > >
>> > > > Thanks for proposing this FLIP. Big +1 for this effort. What others
>> > > > might not see is that we are still in the learning process of how we
>> > > > want to use the PTF framework for our built-in functions.
>> Overloading
>> > > > single functions with too much functionality is definitely an issue
>> that
>> > > > happened with TO/FROM_CHANGELOG. A TO_APPEND would most likely have
>> > > > solved 80% of all use cases, and I agree that we should make the
>> most
>> > > > common case as convenient as possible.
>> > > >
>> > > > As an optimization, we might even be able to remove the
>> > > > ChangelogNormalize if a TO_APPEND immediately follows it. Maybe we
>> can
>> > > > add this to the Future Work section.
>> > > >
>> > > > Thanks,
>> > > > Timo
>> > > >
>> > > >
>> > > > On 08.11.26 07:20, Gustavo de Morais wrote:
>> > > > > Hi everyone,
>> > > > >
>> > > > > I'd like to propose FLIP-621: Add built-in TO_APPEND PTF [1] for
>> > > > > discussion. Since we added TO_CHANGELOG, we've seen the same
>> pattern come
>> > > > > up frequently: users have an updating table and want to write its
>> inserts
>> > > > > and updates to an append-only sink, dropping deletes.
>> TO_CHANGELOG can do
>> > > > > this, but it needs an op_mapping that drops deletes plus a
>> projection
>> > > > that
>> > > > > removes the op column.
>> > > > >
>> > > > > TO_APPEND gives everyone else a very easy way to stay in the
>> append
>> > > > world,
>> > > > > without having to understand the changelog behind it. Full
>> signature:
>> > > > >
>> > > > >       SELECT * FROM TO_APPEND(input => TABLE my_table);
>> > > > >
>> > > > > Inserts and updates are emitted as inserts, update-befores and
>> deletes
>> > > > are
>> > > > > dropped, and the input schema stays unchanged. Following the PTF
>> > > > guidelines
>> > > > > [2], we want to provide functions that solve one problem at a
>> time.
>> > > > > TO_APPEND fits that well: it is a stateless, row-semantics PTF
>> with an
>> > > > easy
>> > > > > to reason implementation. It also covers the use case discussed
>> in the
>> > > > > mailing list for adding one more parameter to TO_CHANGELOG [3].
>> Looking
>> > > > > forward to your feedback and thoughts.
>> > > > >
>> > > > > Kind regards,
>> > > > > Gustavo
>> > > > >
>> > > > > [1]
>> > > > >
>> > > >
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-621%3A+Add+built-in+TO_APPEND+PTF
>> > > > > [2]
>> > > > >
>> > > >
>> https://cwiki.apache.org/confluence/spaces/FLINK/pages/462816208/Built-in+Process+Table+Function+Guidelines
>> > > > > [3]
>> https://lists.apache.org/thread/pfd6ktpyh95j126jm0qolngkrm2bpyhg
>> > > >
>> > > >
>> > >
>>
>

Reply via email to