Re: Nested column types and equality delete files

Created https://github.com/apache/iceberg/pull/8981 to finalize the thread. On Fri, Oct 27, 2023 at 8:55 PM Renjie Liu wrote: > You are right. Null always needs special treatment. I think allowing > null value in equality id is reasonable, but should we treat it as > distinct? PG treats it as

Re: Nested column types and equality delete files

You are right. Null always needs special treatment. I think allowing null value in equality id is reasonable, but should we treat it as distinct? PG treats it as destinct by default, but allows configuration to treat it as no distinct: https://stackoverflow.com/questions/8289100/create-unique-co

Re: Nested column types and equality delete files

> > Iceberg spec has a clear definition of constraints about identifier id > fields . I think > it would make sense if equality id fields share similar constraints. Makes sense, however it appears that for equality delete null values are int

Re: Nested column types and equality delete files

Hi, Micah: Iceberg spec has a clear definition of constraints about identifier id fields . I think it would make sense if equality id fields share similar constraints. On Thu, Oct 26, 2023 at 4:24 AM Micah Kornfield wrote: > Sorry I think

Re: Nested column types and equality delete files

Sorry I think I missed a question: Similarly, I think we could handle fields with primitive or struct types struct types add another dimension of complexity, I'd don't think it is harmful to necessarily support them, but it also doesn't seem like they add a lot of value when compared to enumerat

Re: Nested column types and equality delete files

I think nesting in struct makes sense to support as this is consistent with partitioning input columns. I can propose a PR if there aren't any more opinions here. On Fri, Oct 20, 2023 at 3:49 PM Ryan Blue wrote: > You're right. It calls out that `float` and `double` columns can't be > used, but

Re: Nested column types and equality delete files

You're right. It calls out that `float` and `double` columns can't be used, but there's a question around what is "equal" for maps, at the least. I think the reasonable thing to do is to allow top-level fields and fields that are nested within only struts. Any field nested within a map or list sho