Hey Xuyang, Thanks for the updates and reply!
Regarding dropping the restriction: In my thinking, in Flink's changelog semantics, -D ends the row's lifetime. When we see -D followed by +I for the same PK, like in the example you gave, that's imo creating a new row rather than updating the existing one. I don't think it makes sense to start tracking relations between "conceptually different rows". If it were an update and still the same row, I'd expect a +/-U instead. So I'm still inclining towards being +1 to drop it. That means, NO UPDATE would mean "immutable while the row exists" rather than "immutable for this PK forever". For DeltaJoin's stricter needs (no deletes), we could enforce that separately during planning. Does that distinction make sense to you? Let me know what you think. Kind regards, Gustavo On Thu, 26 Feb 2026 at 05:17, Xuyang <[email protected]> wrote: > Hi Gustavo, > Regarding your suggestion to remove the deletion restriction: it's not > only tied to delta joins. My primary concern before has been the > significant overhead for the storage engine in tracking no update column > changes across separate INSERT and DELETE messages. > Consider this example, where col1 is the primary key and col3 is declared > as a NO UPDATE column: > Schema: (col1, col2, col3) > 1）+I (pk1, a1, b1) > 2）-D (pk1, a1, b1) > 3）+I (pk1, a2, b2) > In this sequence, the value of the no update column col3 changes from b1 > to b2, which violates the NO UPDATE constraint. > If we relax the restriction on DELETE operations, we would effectively > shift the responsibility to users to guarantee that values of no update > columns remain consistent across corresponding INSERT, DELETE, and UPDATE > records. > Given these implications, I’d like to hear your thoughts on whether we > should proceed with removing this restriction. > > > Separately, regarding the additional details you mentioned, I’ve updated > them into the FLIP. Here’s a quick summary: > - If a user declares NO UPDATE (b, c) on a table without a primary key, an > error will be thrown: “NO UPDATE constraints must be defined on tables with > a primary key.” > - If a user declares NO UPDATE(a) and column a is already part of the > primary key, the declaration is silently accepted. > - Updated: CONSTRAINT %s COLUMNS (%s) NO UPDATE%s > > > > -- > > Best！ > Xuyang > > > > At 2026-02-19 20:10:11, "Gustavo de Morais" <[email protected]> > wrote: > >Hey Xuyang, > > > >That's an interesting concept, thanks for the proposal! > > > >I like the FLIP and I think this could open some other optimizations. That > >said, I think it makes sense to remove the deletion restriction from the > >FLIP - since it's mostly a necessity that comes from the DeltaJoin. We > >could make NO UPDATE be about immutability which is not directly connected > >to row permanence. As far as I know, the DeltaJoin already enforces the > >deletion restriction during planning for its sources, so it doesn't have > to > >be enforced by this functionality as well. > > > >Also, some small clarifications that could be added to the FLIP: > >- If someone declares NO UPDATE (b, c) on a table without a primary key. I > >suppose that's an error? > >- If someone declares NO UPDATE(a) and a is already a primary key. Is it > an > >error or do we silently accept it? > >- nit: CONSTRAINT %s FIELDS (%s) NO UPDATE%s -> you mean COLUMNS instead > of > >FIELDS, right? > > > >Kind regards, > >Gustavo > > > > > > > >On Fri, 13 Feb 2026 at 10:08, Xuyang <[email protected]> wrote: > > > >> Hi, everyone. > >> I’d like to propose FLIP-566: Introduce a new NO UPDATE column > >> constraint[1]. > >> Flink has introduced the Delta Join, whose core advantage lies in > >> replacing redundant local state storage with direct queries to external > >> storage systems (e.g., Apache Fluss). It currently relies on the upsert > >> key, which ensures correct changelog processing without UPDATE_BEFORE > >> messages. But this assumes the join key must be part of the primary key. > >> As modern storage systems increasingly support general-purpose secondary > >> secondary indexes (not limited to primary keys), this restriction is > >> becoming outdated. We need a new semantic mechanism to guarantee the > >> immutability of the join key—specifically, that for a given primary key, > >> the column values comprising the join key cannot be modified. > >> Looking forward to your feedback. > >> > >> > >> [1] > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-566%3A+Introduce+a+new+NO+UPDATE+constraint > >> > >> > >> > >> > >> > >> -- > >> > >> Best！ > >> Xuyang >
