Hey Y'all!
Here are the notes and recording for the Jan 15th meeting! Video - https://www.youtube.com/watch?v=9ZLWQSZvLIw Notes - https://docs.google.com/document/d/1YuGhUdukLP5gGiqCbk0A5_Wifqe2CZWgOd3TbhY3UQg/edit?tab=t.0 Notes : --- - Highlights - Java - Hive Runtime is now part of the Hive project itself (Thanks Manu!) - Support for removing unused partition specs <https://github.com/apache/iceberg/pull/10755> (Thanks Xianjin!) - Added Internal writers for Avro (Thanks, Ajantha!) - FastAppend allows adding files with different specs <https://github.com/apache/iceberg/pull/11771> (Thanks, Anurag!) - Spark: Implemented RewriteTablePath (Thanks, Szehon!) - Spark: Updated delete granularity to file (Thanks, Amogh!) - Spark: Implemented v3 DVs (Thanks, Amogh!) - Spark: Implemented default values for Parquet/Avro - Rust - Iceberg-Rust 0.4.0 <https://github.com/apache/iceberg-rust/releases/tag/v0.4.0> release is out (thanks Sung!) - Added support for parsing Puffin FileMetadata <https://github.com/apache/iceberg-rust/pull/765> (Thanks Farooq!) - Add Manifest metadata table <https://github.com/apache/iceberg-rust/pull/861/> (Thanks Xuanwo) - PyIceberg - We now have bucketing partition support thanks to the dependency on Iceberg-Rust 0.4.0 (pyiceberg-core <https://pypi.org/project/pyiceberg-core/>) - Add support for pluggable LocationProviders <https://github.com/apache/iceberg-python/pull/1452> (thanks Smaheshwar) - Add Support for Dynamic Overwrite <https://github.com/apache/iceberg-python/pull/931> (thanks Adrian) - Support multiple S3 regions <https://github.com/apache/iceberg-python/pull/1453> and multiple FileIOs (thanks jiakai) - PyIceberg 0.8.1 release! 🎉 - Go - Added remaining table updates/requirements for REST catalog (Thanks, jwtryg) - Added catalog registry / improved APIs around catalog creation (Thanks, Matt) - Improved manifest scanning & concurrency (Thanks, Kaushik) - C++ - Scaffolding is up (CI <https://github.com/apache/iceberg-cpp/pull/7>, Linter <https://github.com/apache/iceberg-cpp/pull/16> and License <https://github.com/apache/iceberg-cpp/pull/10> checker), feel free to join in on the discussion of the project MVP <https://github.com/apache/iceberg-cpp/issues/2>. - Releases - 1.7.2 <https://lists.apache.org/thread/h99gnh4kc4g7qcprq36mlxnm8xl3w37h> release is almost ready for the first RC - 1.8.0 <https://lists.apache.org/thread/h99gnh4kc4g7qcprq36mlxnm8xl3w37h> - Amogh will be RM - 1.9.0 <https://github.com/apache/iceberg/milestone/53>- Ajantha RM - Variant - Possibly partition stats - REST Auth - Suggested agenda items: - PSA: License and notice requirements (Related issue: #11559 <https://github.com/apache/iceberg/issues/11559>) - Recording Responsibilities - Iceberg Summit - April 8th In Person, 9th Virtual - Hyatt Regency San Francisco Downtown SOMA - Iceberg Summit CFP 2025 <https://sessionize.com/iceberg-summit-2025/> - Share and submit! - Iceberg Summit Travel Grants in Progress - Spec changes in row lineage <https://github.com/apache/iceberg/pull/11976> - Iceberg Metadata for Variant Shredding <https://docs.google.com/document/d/1gAvt0x_ez89O8y-YqkCdMnTEykb-583YslYOgzf5sPg/edit?tab=t.0> - What would the community think about creating a pluggable FileFormat to try out new formats, like Vortex, Lance - Document Snapshot Summary Optional fields - Vote is out: https://lists.apache.org/thread/0jqfryvx0m19698h9xph9co0wh72hpv3 - Main body versus appendix? Thanks everyone! See you next time!
