Re: Duplicates are getting inserted into Iceberg tables even after de-duplication

I do recall an issue where duplicate data/delete files where possible, but I'm not sure if that's the underlying cause in your case. The issue was fixed by #10007 and was shipped with Iceberg 1.6.0. On Thu, Nov 7, 2024 at 11:12 PM Lewis, William wrot

RE: Duplicates are getting inserted into Iceberg tables even after de-duplication

On 2024/03/13 22:38:06 Shwetha Dharmarajan wrote: > We are using Apache Iceberg with AWS Glue. We are seeing an issue where > duplicates are getting inserted into the table, even after making sure there > are no duplicates in the data being upserted into the table. We use MERGE sql > to upsert d