Hey all, Here are the meeting minutes from the meeting just before the holiday.
https://www.youtube.com/watch?v=OwyBlUi2CRc - Highlights - Encryption: Added StandardEncryptionManager(https://github.com/apache/iceberg/pull/6884) (Thanks, Gidon!) - Views: Added support in REST catalog (Thanks, Eduard!) - Views: Added support in Nessie catalog (Thanks, Ajantha!) - Added support for spec ID in rewrite_manifests(https://github.com/apache/iceberg/pull/9242) procedure (Thanks, Pucheng!) - Java: Flink - add watermark alignment support to FLIP-27 source #8553(https://github.com/apache/iceberg/pull/8553) (Thanks, Peter!) - Rust: - Working toward a first release! Documentation(https://github.com/apache/iceberg-rust/issues/114) (how-to-install, examples, how-to-release(https://github.com/apache/iceberg-rust/issues/81)) and final tests(https://github.com/apache/iceberg-rust/issues/70) are still pending. - Merged the manifest-list and manifest reader/writer(https://github.com/apache/iceberg-rust/pull/79) today (Thanks, Jan, Renjie, Xuanwo) - Releases - PyIceberg 0.6.0(https://github.com/apache/iceberg-python/milestone/1) release with write support - Java 1.5.0 - January - Delete executor cache (Spark) - Flink 1.18 support (merged!) - compatibility broken from Flink - FLIP-27 source by default - View support in Spark 3.5 - Remove spark 3.2 folder (It was deprecated in the previous release) ? - Java 2.0.0 - Upgrade to a newer JDK version (JDK 17)? - Drop Hive from the source tree? - ResolvingFileIO as the default (https://github.com/apache/iceberg/pull/8272) - Discussion - REST catalog scan planning - Sharding - Pagination - Require sharding? - REST catalog append (and other operations?) - REST catalog planning with other formats (Hive, Delta) - Name mapping - Fixing ID fallback - Hive catalog views AI-generated chapter summaries: 0:00 Introduction 6:09 Good progress being made on Rust implementation, targeting first release soon 17:49 Targeting Spark view support for Iceberg Java 1.5 release in January 21:30 Discussed plans for Iceberg Java 2.0 to clean up deprecated code 27:57 Discussed REST API proposals for scan planning and other operations 46:37 Discussed best way to represent Iceberg views in Hive catalog Topics Rust Implementation Update - 6:09 Rust implementation making good progress, manifest list and reader/writer code was merged recently - 6:23 Targeting first Rust release soon that can load tables and do some scan planning - 7:04 Documentation will be on Rust side initially but can migrate later Iceberg Python 0.6 Release Planning - 6:23 Working on getting write support in for 0.6 release - 8:08 Snapshot regeneration was a big step - 8:33 Call for any other items people want to get into 0.6 Iceberg Java 1.5 Release Planning - 8:49 Targeting release in January, about 2 months since last release - 17:49 Want to get Spark view support in, though won't block release on it - 20:39 Also want to drop Spark 3.2 support - 14:27 Planning to make Flip-27 the default file format Iceberg Java 2.0 Planning - 21:30 Main goal is to clean up deprecated code - 22:54 Also want to remove unsafe ID fallback resolution - 24:09 Likely keep spec v3 work separate for Java 3.0 release REST API Proposals - 27:57 Discussed scan planning proposal to break up planning into shards - 27:57 Allows parallel planning on server and client side - 27:57 Some concerns around pagination and making it fully stateless
