For rust, we have added support parquet data file writer, and support for other writers are undergoing.
On Thu, Dec 12, 2024 at 9:26 AM Gang Wu <ust...@gmail.com> wrote: > For C++, I think it is aimed for a full featured C++ library (not for > puffin implementation only). > > On Thu, Dec 12, 2024 at 6:14 AM rdb...@gmail.com <rdb...@gmail.com> wrote: > >> I'll update it. Thanks! >> >> (By the way, the Avro default value support was in the Java section) >> >> On Wed, Dec 11, 2024 at 2:00 PM Matt Topol <zotthewiz...@gmail.com> >> wrote: >> >>> For the Go release, can we please point out that it supports reading the >>> data too, not just metadata? >>> >>> It produces a stream of Arrow record batches. >>> >>> On Wed, Dec 11, 2024, 4:22 PM Walaa Eldin Moustafa < >>> wa.moust...@gmail.com> wrote: >>> >>>> Hi Ryan, >>>> >>>> For Table Format V3, we could point out that the default value support >>>> for Avro has been merged and support for other formats is ongoing. >>>> >>>> Thanks, >>>> Walaa. >>>> >>>> >>>> On Wed, Dec 11, 2024 at 12:51 PM rdb...@gmail.com <rdb...@gmail.com> >>>> wrote: >>>> >>>>> Hi everyone, >>>>> >>>>> It’s time to report to the board again. Great to see all the progress >>>>> here, and awesome to have our first go release this quarter! >>>>> >>>>> My draft is below. Please reply if there’s anything you’d like to add >>>>> or change. Thanks! >>>>> >>>>> Ryan >>>>> Description: >>>>> >>>>> Apache Iceberg is a table format for huge analytic datasets that is >>>>> designed >>>>> for high performance and ease of use. >>>>> Project Status: >>>>> >>>>> Current project status: Ongoing >>>>> Issues for the board: None >>>>> Membership Data: >>>>> >>>>> Apache Iceberg was founded 2020-05-19 (5 years ago) >>>>> There are currently 32 committers and 21 PMC members in this project. >>>>> The Committer-to-PMC ratio is roughly 4:3. >>>>> >>>>> Community changes, past quarter: >>>>> >>>>> - No new PMC members. Last addition was Amogh Jahagirdar on >>>>> 2024-08-12. >>>>> - Matthew Topol was added as committer on 2024-12-09 >>>>> - Scott Donnelly was added as committer on 2024-12-10 >>>>> >>>>> Project Activity: >>>>> >>>>> Releases >>>>> >>>>> - 1.7.1 was released on 2024-12-06. >>>>> - 1.7.0 was released on 2024-11-08. >>>>> - PyIceberg 0.8.1 was released on 2024-12-06. >>>>> - PyIceberg 0.8.0 was released on 2024-11-18. >>>>> - Go 0.1.0 was released on 2024-11-18. >>>>> >>>>> Table format (v3) >>>>> >>>>> - Added deletion vectors and synchronous maintenance to improve >>>>> row-level ops >>>>> - Added row lineage fields and requirements for fine-grained row >>>>> tracking >>>>> - Proposal for geography and geometry types is close to consensus >>>>> - Update to add Parquet’s variant type is approved, waiting on >>>>> Parquet upstream >>>>> - Finalized new type promotion rules >>>>> >>>>> Puffin format >>>>> >>>>> - Added deletion vector blob type to support DVs in tables >>>>> >>>>> REST catalog spec >>>>> >>>>> - Added storage credentials passing >>>>> - Added credential refresh >>>>> - Created a docker image for catalog testing >>>>> - Discussing proposal for partial metadata commits >>>>> - Discussed partial metadata loading >>>>> >>>>> Views >>>>> >>>>> - Discussions about materialized view metadata are ongoing >>>>> >>>>> Java >>>>> >>>>> - Released new Kafka Connect sink >>>>> - Added default values implementation for Avro >>>>> - Added nanosecond timestamps >>>>> - Added v3 DV support in core, ongoing work in Spark >>>>> - Flink: Made FLIP-27 source the default >>>>> - Spark: Removed Spark 3.3 support >>>>> - Hive: Removing Hive 2.x and 3.x (Iceberg support is in Hive for >>>>> 4.x and on) >>>>> - Pig: Removed the iceberg-pig module that is no longer used >>>>> >>>>> PyIceberg >>>>> >>>>> - Support: Added Python 3.12, dropped Python 3.8 >>>>> >>>>> Rust >>>>> >>>>> - Support for default values and type promotion in reads >>>>> - Added TableMetadataBuilder >>>>> - Implemented table requirements >>>>> >>>>> Go >>>>> >>>>> - Produced the first go release! >>>>> - Supports scan planning and reading metadata >>>>> - Supports loading and listing tables with the Glue catalog >>>>> - Supports local and S3 storage >>>>> >>>>> C++ >>>>> >>>>> - Added a C++ repository for a Puffin implementation >>>>> >>>>> Community Health: >>>>> >>>>> The PMC has published guidelines for contributors that want to know >>>>> more about >>>>> how they can become committers on the Iceberg site. This guide should >>>>> help >>>>> contributors understand how Iceberg and other ASF communities decide >>>>> and add >>>>> committers, and should set expectations clearly. This was the most >>>>> important >>>>> follow up from discussions on the dev list earlier this year, where it >>>>> became >>>>> clear that contributors did not understand the requirements or process. >>>>> >>>>> The community has started planning a second Iceberg Summit, intended >>>>> to be held >>>>> in Spring of 2025. The proposal details are being finalized (such as >>>>> the members >>>>> of the selection committee) and will be submitted for approval in the >>>>> next few >>>>> weeks. >>>>> >>>>> The community added two new committers this quarter and had a slight >>>>> increase in >>>>> the number of contributors. >>>>> >>>>> There were also a number of commercial announcements from companies >>>>> adding or >>>>> expanding support for Iceberg. >>>>> >>>>
