Re: Updating the Iceberg table architecture diagram

Hi I think we have to keep it "clear and simple" as possible. I would prefer to have one diagram per spec version (to be clear in the scope). So, I would rather keep the current diagram (working for v1) and add a new one v2 centric. It would be great to have a "side by side" presentation, somethi

Re: Updating the Iceberg table architecture diagram

I'm not sure that an update to the diagram is needed. Adding stats files and delete files makes the diagram harder to follow. And its purpose is to clearly show the high-level structure of the format to help people understand the basic idea, not to show all of the details. I think more complexity u

Re: Updating the Iceberg table architecture diagram

since v2 has been out for a while and most tools that support iceberg support v2 (not to mention some only support v2), I think having a single diagram and using dotted lines for the delete manifests and delete files will cause more confusion than benefit. also because of the support and adoption o

Re: Updating the Iceberg table architecture diagram

> > However, there are a lot of boxes and new terms. What do you think of > keeping both files, and indicating that the old applies to V1 tables, and > the new one to V2 tables. Statistics are common for both V1 and V2. So, we can't say old applies to V1 and new applies to V2. For delete, we are

Re: Updating the Iceberg table architecture diagram

Thanks for updating the diagram and +1 to Fokko's suggestion. On Fri, Nov 3, 2023 at 3:43 PM Fokko Driesprong wrote: > Hey Jason, thanks for updating the chart. > > I like it a lot. However, there are a lot of boxes and new terms. What do > you think of keeping both files, and indicating that th

Re: Updating the Iceberg table architecture diagram

Hey Jason, thanks for updating the chart. I like it a lot. However, there are a lot of boxes and new terms. What do you think of keeping both files, and indicating that the old applies to V1 tables, and the new one to V2 tables. Kind regards, Fokko Op vr 3 nov 2023 om 14:37 schreef Aaron Niskode

Re: Updating the Iceberg table architecture diagram

An update would be greatly appreciated, thank you! On Thu, Nov 2, 2023 at 12:42 PM Jason Hughes wrote: > Hey all, > > The current architecture diagram > for an iceberg > table hasn't been updated in over 3 years, and there's are some aspects

Updating the Iceberg table architecture diagram

Hey all, The current architecture diagram for an iceberg table hasn't been updated in over 3 years, and there's are some aspects to the architecture of an iceberg table that have changed, most notably delete files and puffin files. since this d