Hi all,

Variant shredding writers now exist across iceberg-java, arrow-go, and
arrow-rs, and shredded write paths are landing soon in iceberg-go,
iceberg-rust, and (future) iceberg-python. We don't yet have a shared
policy on whether to shred, how the layout is declared, and how it's
expressed so every implementation produces compatible output.

I'd like to propose one: Variant Shredding Policy Across Iceberg
Implementations
<https://docs.google.com/document/d/1OaB45mg9JJ_JRg-ipuvyh4AtOmaGmdWMsy_mJKiu5xs/edit?usp=sharing>
.
Please take a look.

Next steps:
  1. The community reviews the proposal and agrees on the shredding
representation.
  2. Once consensus is reached, I'll start a [VOTE] thread.
  3. After the vote passes, I'll land the adjustments in the Java
implementation.
  4. iceberg-go, iceberg-rust, and others follow to align.

Thanks,
Neelesh

Reply via email to