Apologies, this should fix the formatting issues. Iceberg Community Sync (Recorded) - November 22 VIEW RECORDING: https://www.youtube.com/watch?v=iz0Oex1hQA0 Meeting Purpose:
Weekly Iceberg dev sync meeting to discuss recent updates, issues, and next steps Key Takeaways - No urgent need for 1.4.3 release currently, will wait for meaningful bug fixes - Branches should reflect current table schema, not schema at time of branch snapshot - Need to fail time travel queries on branches since history not maintained Topics: Recent Updates - Good improvements to delete performance from Anton and others - Added ability to filter stats after planning to avoid deserializing unneeded column stats - Added metrics to Spark planning phase - Progress on REST API support in Rust and Python - Table metadata updates progressing in Python Iceberg 1.4.3 Release - Avro CVE mitigation not needed for Iceberg, no urgent need for 1.4.3 currently - Will wait for merge consistency bug fix before next patch release Dependency Version Upgrades - Avoid major version bumps unless shaded and proven reliable - Prefer major upgrades in major releases, minor in minor, patches in patches - Watch for breaking changes even in patch releases - Prioritize avoiding downstream disruption over staying on latest versions Branch Schema Behavior - Branches should reflect current table schema like main branch - Makes sense for tags to preserve schema at tag time - Will update behavior so branches use current table schema Time Travel on Branches - As of time not supported on branches currently since history not maintained - Should fail time travel queries on branches - Will add test and update behavior Next Steps: - Release 1.4.3 after meaningful bug fixes - Review PR updating branch schema behavior - Add test and fix for time travel on branches On Thu, Dec 7, 2023 at 4:48 PM Brian Olsen <bitsondata...@gmail.com> wrote: > > Key Takeaways 0:00 Introduction 4:33 No urgent need for 1.4.3 release > currently, will wait for meaningful bug fixes 25:43 Branches should reflect > current table schema, not schema at time of branch snapshot 34:13 Need to > fail time travel queries on branches since history not maintained Topics > Recent Updates 0:17 Good improvements to delete performance from Anton and > others 1:16 Added ability to filter stats after planning to avoid > deserializing unneeded column stats 2:26 Added metrics to Spark planning > phase 2:50 Progress on REST API support in Rust and Python 3:44 Table > metadata updates progressing in Python Iceberg 1.4.3 Release 5:09 Avro CVE > mitigation not needed for Iceberg, no urgent need for 1.4.3 currently 4:28 > Will wait for merge consistency bug fix before next patch release Dependency > Version Upgrades 14:08 Avoid major version bumps unless shaded and proven > reliable 14:45 Prefer major upgrades in major releases, minor in minor, > patches in patches 15:14 Watch for breaking changes even in patch releases > 21:38 Prioritize avoiding downstream disruption over staying on latest > versions Branch Schema Behavior 34:26 Branches should reflect current table > schema like main branch 24:28 Makes sense for tags to preserve schema at tag > time 24:03 Will update behavior so branches use current table schema Time > Travel on Branches 33:00 As of time not supported on branches currently since > history not maintained 34:09 Should fail time travel queries on branches > 34:17 Will add test and update behavior Next Steps: 4:23 Release 1.4.3 after > meaningful bug fixes 24:06 Review PR updating branch schema behavior 34:13 > Add test and fix for time travel on branches
