Hi Ryan, Thanks for putting together the report!
I have a couple of items that might be helpful to include. PyIceberg * Updated the release process and documentation * Updated integration tests to use the TCK REST catalog docker image (`apache/iceberg-rest-fixture`) built using the apache/iceberg repo * Added manifest files caching * Added support for all metadata tables via the Table Inspect API * Added support for High Availability mode for Hive Metastore * Removed `numpy` as a hard dependency * Pyiceberg crossed 100k daily downloads on PyPi ( https://pypistats.org/packages/pyiceberg) Community * Several reoccurring community meetups have started in Seattle/SF/Singapore and more are planned * Meetup presentations are recorded and available on the Apache Iceberg Meetup YouTube channel (https://www.youtube.com/@IcebergMeetup) Best, Kevin Liu On Wed, Dec 11, 2024 at 10:30 PM Péter Váry <peter.vary.apa...@gmail.com> wrote: > Hi Ryan, > Thanks for putting this together! > For Java/Flink we could mention that ExpireSnapshots TableMaintenance is > available now. > > On Thu, Dec 12, 2024, 04:47 Ajantha Bhat <ajanthab...@gmail.com> wrote: > >> At Java side, I would add >> >> - Core util to compute partition stats has been merged. >> https://github.com/apache/iceberg/pull/11146 >> >> - REST catalog TCK has been merged and docker image is published under ` >> >> *apache/iceberg-rest-fixture`*Also, >> >>> Spark: Removed Spark 3.3 support >> >> We just deprecated it after 1.7.0, so there will be one last release in >> 1.8.0. >> Maybe we can reword as Deprecated instead of removed as the code is still >> there. >> >> - Ajantha >> >> On Thu, Dec 12, 2024 at 7:25 AM Renjie Liu <liurenjie2...@gmail.com> >> wrote: >> >>> For rust, we have added support parquet data file writer, and support >>> for other writers are undergoing. >>> >>> >>> On Thu, Dec 12, 2024 at 9:26 AM Gang Wu <ust...@gmail.com> wrote: >>> >>>> For C++, I think it is aimed for a full featured C++ library (not for >>>> puffin implementation only). >>>> >>>> On Thu, Dec 12, 2024 at 6:14 AM rdb...@gmail.com <rdb...@gmail.com> >>>> wrote: >>>> >>>>> I'll update it. Thanks! >>>>> >>>>> (By the way, the Avro default value support was in the Java section) >>>>> >>>>> On Wed, Dec 11, 2024 at 2:00 PM Matt Topol <zotthewiz...@gmail.com> >>>>> wrote: >>>>> >>>>>> For the Go release, can we please point out that it supports reading >>>>>> the data too, not just metadata? >>>>>> >>>>>> It produces a stream of Arrow record batches. >>>>>> >>>>>> On Wed, Dec 11, 2024, 4:22 PM Walaa Eldin Moustafa < >>>>>> wa.moust...@gmail.com> wrote: >>>>>> >>>>>>> Hi Ryan, >>>>>>> >>>>>>> For Table Format V3, we could point out that the default value >>>>>>> support for Avro has been merged and support for other formats is >>>>>>> ongoing. >>>>>>> >>>>>>> Thanks, >>>>>>> Walaa. >>>>>>> >>>>>>> >>>>>>> On Wed, Dec 11, 2024 at 12:51 PM rdb...@gmail.com <rdb...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi everyone, >>>>>>>> >>>>>>>> It’s time to report to the board again. Great to see all the >>>>>>>> progress here, and awesome to have our first go release this quarter! >>>>>>>> >>>>>>>> My draft is below. Please reply if there’s anything you’d like to >>>>>>>> add or change. Thanks! >>>>>>>> >>>>>>>> Ryan >>>>>>>> Description: >>>>>>>> >>>>>>>> Apache Iceberg is a table format for huge analytic datasets that is >>>>>>>> designed >>>>>>>> for high performance and ease of use. >>>>>>>> Project Status: >>>>>>>> >>>>>>>> Current project status: Ongoing >>>>>>>> Issues for the board: None >>>>>>>> Membership Data: >>>>>>>> >>>>>>>> Apache Iceberg was founded 2020-05-19 (5 years ago) >>>>>>>> There are currently 32 committers and 21 PMC members in this >>>>>>>> project. >>>>>>>> The Committer-to-PMC ratio is roughly 4:3. >>>>>>>> >>>>>>>> Community changes, past quarter: >>>>>>>> >>>>>>>> - No new PMC members. Last addition was Amogh Jahagirdar on >>>>>>>> 2024-08-12. >>>>>>>> - Matthew Topol was added as committer on 2024-12-09 >>>>>>>> - Scott Donnelly was added as committer on 2024-12-10 >>>>>>>> >>>>>>>> Project Activity: >>>>>>>> >>>>>>>> Releases >>>>>>>> >>>>>>>> - 1.7.1 was released on 2024-12-06. >>>>>>>> - 1.7.0 was released on 2024-11-08. >>>>>>>> - PyIceberg 0.8.1 was released on 2024-12-06. >>>>>>>> - PyIceberg 0.8.0 was released on 2024-11-18. >>>>>>>> - Go 0.1.0 was released on 2024-11-18. >>>>>>>> >>>>>>>> Table format (v3) >>>>>>>> >>>>>>>> - Added deletion vectors and synchronous maintenance to improve >>>>>>>> row-level ops >>>>>>>> - Added row lineage fields and requirements for fine-grained >>>>>>>> row tracking >>>>>>>> - Proposal for geography and geometry types is close to >>>>>>>> consensus >>>>>>>> - Update to add Parquet’s variant type is approved, waiting on >>>>>>>> Parquet upstream >>>>>>>> - Finalized new type promotion rules >>>>>>>> >>>>>>>> Puffin format >>>>>>>> >>>>>>>> - Added deletion vector blob type to support DVs in tables >>>>>>>> >>>>>>>> REST catalog spec >>>>>>>> >>>>>>>> - Added storage credentials passing >>>>>>>> - Added credential refresh >>>>>>>> - Created a docker image for catalog testing >>>>>>>> - Discussing proposal for partial metadata commits >>>>>>>> - Discussed partial metadata loading >>>>>>>> >>>>>>>> Views >>>>>>>> >>>>>>>> - Discussions about materialized view metadata are ongoing >>>>>>>> >>>>>>>> Java >>>>>>>> >>>>>>>> - Released new Kafka Connect sink >>>>>>>> - Added default values implementation for Avro >>>>>>>> - Added nanosecond timestamps >>>>>>>> - Added v3 DV support in core, ongoing work in Spark >>>>>>>> - Flink: Made FLIP-27 source the default >>>>>>>> - Spark: Removed Spark 3.3 support >>>>>>>> - Hive: Removing Hive 2.x and 3.x (Iceberg support is in Hive >>>>>>>> for 4.x and on) >>>>>>>> - Pig: Removed the iceberg-pig module that is no longer used >>>>>>>> >>>>>>>> PyIceberg >>>>>>>> >>>>>>>> - Support: Added Python 3.12, dropped Python 3.8 >>>>>>>> >>>>>>>> Rust >>>>>>>> >>>>>>>> - Support for default values and type promotion in reads >>>>>>>> - Added TableMetadataBuilder >>>>>>>> - Implemented table requirements >>>>>>>> >>>>>>>> Go >>>>>>>> >>>>>>>> - Produced the first go release! >>>>>>>> - Supports scan planning and reading metadata >>>>>>>> - Supports loading and listing tables with the Glue catalog >>>>>>>> - Supports local and S3 storage >>>>>>>> >>>>>>>> C++ >>>>>>>> >>>>>>>> - Added a C++ repository for a Puffin implementation >>>>>>>> >>>>>>>> Community Health: >>>>>>>> >>>>>>>> The PMC has published guidelines for contributors that want to know >>>>>>>> more about >>>>>>>> how they can become committers on the Iceberg site. This guide >>>>>>>> should help >>>>>>>> contributors understand how Iceberg and other ASF communities >>>>>>>> decide and add >>>>>>>> committers, and should set expectations clearly. This was the most >>>>>>>> important >>>>>>>> follow up from discussions on the dev list earlier this year, where >>>>>>>> it became >>>>>>>> clear that contributors did not understand the requirements or >>>>>>>> process. >>>>>>>> >>>>>>>> The community has started planning a second Iceberg Summit, >>>>>>>> intended to be held >>>>>>>> in Spring of 2025. The proposal details are being finalized (such >>>>>>>> as the members >>>>>>>> of the selection committee) and will be submitted for approval in >>>>>>>> the next few >>>>>>>> weeks. >>>>>>>> >>>>>>>> The community added two new committers this quarter and had a >>>>>>>> slight increase in >>>>>>>> the number of contributors. >>>>>>>> >>>>>>>> There were also a number of commercial announcements from companies >>>>>>>> adding or >>>>>>>> expanding support for Iceberg. >>>>>>>> >>>>>>>
