Re: Duplicates are getting inserted into Iceberg tables even after de-duplication

I do recall an issue where duplicate data/delete files where possible, but I'm not sure if that's the underlying cause in your case. The issue was fixed by #10007 and was shipped with Iceberg 1.6.0. On Thu, Nov 7, 2024 at 11:12 PM Lewis, William wrot

RE: Duplicates are getting inserted into Iceberg tables even after de-duplication

On 2024/03/13 22:38:06 Shwetha Dharmarajan wrote: > We are using Apache Iceberg with AWS Glue. We are seeing an issue where > duplicates are getting inserted into the table, even after making sure there > are no duplicates in the data being upserted into the table. We use MERGE sql > to upsert d

Duplicates are getting inserted into Iceberg tables even after de-duplication

Hello, We are using Apache Iceberg with AWS Glue. We are seeing an issue where duplicates are getting inserted into the table, even after making sure there are no duplicates in the data being upserted into the table. We use MERGE sql to upsert data into the table. We also see an issue where dup