[
https://issues.apache.org/jira/browse/HIVE-29815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Raghav Aggarwal updated HIVE-29815:
-----------------------------------
Description:
In HIVE-29445, support for Iceberg V3 Row Lineage (ROW__LINEAGE__ID and
LAST__UPDATED__SEQUENCE__NUMBER) was introduced to track CDC mutations across
INSERT, UPDATE, and MERGE operations.
However, support was missed for *DELETE* operations when the table is
explicitly configured with _'write.delete.mode'='copy-on-write'_
*This completely breaks CDC downstream. When a single row is deleted using
Copy-On-Write, Iceberg rewrites the entire file to remove that row. Because the
lineage IDs of the surviving rows are not preserved during this rewrite, they
are all assigned brand new lineage IDs and new sequence numbers.*
> Iceberg: [V3] Support ROW LINEAGE in Copy-On-Write DELETE operations
> --------------------------------------------------------------------
>
> Key: HIVE-29815
> URL: https://issues.apache.org/jira/browse/HIVE-29815
> Project: Hive
> Issue Type: Improvement
> Reporter: Raghav Aggarwal
> Assignee: Raghav Aggarwal
> Priority: Major
>
> In HIVE-29445, support for Iceberg V3 Row Lineage (ROW__LINEAGE__ID and
> LAST__UPDATED__SEQUENCE__NUMBER) was introduced to track CDC mutations across
> INSERT, UPDATE, and MERGE operations.
> However, support was missed for *DELETE* operations when the table is
> explicitly configured with _'write.delete.mode'='copy-on-write'_
>
> *This completely breaks CDC downstream. When a single row is deleted using
> Copy-On-Write, Iceberg rewrites the entire file to remove that row. Because
> the lineage IDs of the surviving rows are not preserved during this rewrite,
> they are all assigned brand new lineage IDs and new sequence numbers.*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
