Hi Gustavo,
Regarding your suggestion to remove the deletion restriction: it's not only
tied to delta joins. My primary concern before has been the significant
overhead for the storage engine in tracking no update column changes across
separate INSERT and DELETE messages.
Consider this example, where col1 is the primary key and col3 is declared as a
NO UPDATE column:
Schema: (col1, col2, col3)
1）+I (pk1, a1, b1)
2）-D (pk1, a1, b1)
3）+I (pk1, a2, b2)
In this sequence, the value of the no update column col3 changes from b1 to b2,
which violates the NO UPDATE constraint.
If we relax the restriction on DELETE operations, we would effectively shift
the responsibility to users to guarantee that values of no update columns
remain consistent across corresponding INSERT, DELETE, and UPDATE records.
Given these implications, I’d like to hear your thoughts on whether we should
proceed with removing this restriction.
Separately, regarding the additional details you mentioned, I’ve updated them
into the FLIP. Here’s a quick summary:
- If a user declares NO UPDATE (b, c) on a table without a primary key, an
error will be thrown: “NO UPDATE constraints must be defined on tables with a
primary key.”
- If a user declares NO UPDATE(a) and column a is already part of the primary
key, the declaration is silently accepted.
- Updated: CONSTRAINT %s COLUMNS (%s) NO UPDATE%s
--
Best！
Xuyang
At 2026-02-19 20:10:11, "Gustavo de Morais" <[email protected]> wrote:
>Hey Xuyang,
>
>That's an interesting concept, thanks for the proposal!
>
>I like the FLIP and I think this could open some other optimizations. That
>said, I think it makes sense to remove the deletion restriction from the
>FLIP - since it's mostly a necessity that comes from the DeltaJoin. We
>could make NO UPDATE be about immutability which is not directly connected
>to row permanence. As far as I know, the DeltaJoin already enforces the
>deletion restriction during planning for its sources, so it doesn't have to
>be enforced by this functionality as well.
>
>Also, some small clarifications that could be added to the FLIP:
>- If someone declares NO UPDATE (b, c) on a table without a primary key. I
>suppose that's an error?
>- If someone declares NO UPDATE(a) and a is already a primary key. Is it an
>error or do we silently accept it?
>- nit: CONSTRAINT %s FIELDS (%s) NO UPDATE%s -> you mean COLUMNS instead of
>FIELDS, right?
>
>Kind regards,
>Gustavo
>
>
>
>On Fri, 13 Feb 2026 at 10:08, Xuyang <[email protected]> wrote:
>
>> Hi, everyone.
>> I’d like to propose FLIP-566: Introduce a new NO UPDATE column
>> constraint[1].
>> Flink has introduced the Delta Join, whose core advantage lies in
>> replacing redundant local state storage with direct queries to external
>> storage systems (e.g., Apache Fluss). It currently relies on the upsert
>> key, which ensures correct changelog processing without UPDATE_BEFORE
>> messages. But this assumes the join key must be part of the primary key.
>> As modern storage systems increasingly support general-purpose secondary
>> secondary indexes (not limited to primary keys), this restriction is
>> becoming outdated. We need a new semantic mechanism to guarantee the
>> immutability of the join key—specifically, that for a given primary key,
>> the column values comprising the join key cannot be modified.
>> Looking forward to your feedback.
>>
>>
>> [1]
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-566%3A+Introduce+a+new+NO+UPDATE+constraint
>>
>>
>>
>>
>>
>> --
>>
>> Best！
>> Xuyang
