Hi community,

I drafted a design for supporting create_from_lake on existing Iceberg tables:

https://docs.google.com/document/d/1REmpQ2r5W12OvW2CU9tNjDWjpkFJegiTZnE1IZbBH_U/edit?tab=t.0

The proposal follows the existing Paimon create_from_lake design as much as 
possible, and only introduces Iceberg-specific deltas where needed.

The main difference is how the target Fluss table type is derived from Iceberg 
metadata:

  *   Iceberg format v1 tables are mapped to Fluss log tables.
  *   Iceberg format v2 tables without identifier fields are mapped to Fluss 
log tables.
  *   Iceberg format v2 tables with non-empty identifier fields are mapped to 
Fluss primary-key tables and reuse the existing bootstrap flow.

For primary-key tables, the design reuses the same coordinator lifecycle, 
single holdPartition model, Tiering Service bootstrap pipeline, snapshot commit 
flow, partition activation, and recovery behavior from the Paimon design. The 
Iceberg-specific part is mainly metadata extraction and using the Iceberg 
reader to read snapshot-visible rows during bootstrap.

Could you please help review the draft and share feedback, especially on:

  *   Whether the table type mapping based on Iceberg format version and 
identifier fields is reasonable.
  *   Whether relying on the Iceberg reader for equality/position delete 
handling during bootstrap is the right boundary.
  *   Whether there are any Iceberg-specific cases missing from the current 
design.

Thanks

Reply via email to