Nice summary, Renjie! Great to see the community grows and welcome new contributors!
Yufei On Wed, Mar 27, 2024 at 7:34 PM Renjie Liu <liurenjie2...@gmail.com> wrote: > Sorry, I missed Himadri Pal for rest catalog: > > 3. Some enhancement of rest catalog, such as oauth, custom headers. Thanks > Himadri Pal(himadripal), whynick1, TennyZhuang! > > On Thu, Mar 28, 2024 at 10:19 AM Renjie Liu <liurenjie2...@gmail.com> > wrote: > >> Hi: >> >> Sorry I couldn't join the meeting due to timezone constraints. >> >> I want to add some summary about progress of iceberg-rust: >> >> 1. We have added hive catalog support(except update table methods). >> Thanks Xuanwo and Marvin Lanhenke(github id: marvinlanhenke)! >> 2. The glue catalog is in progress. Thanks Marvin Lanhenke(github >> id: marvinlanhenke)! >> 3. Some enhancement of rest catalog, such as oauth, custom headers. >> Thanks whynick1, TennyZhuang! >> 4. The community is actively working on reading support, including >> reading to arrow, filter support, please stay tuned! >> 5. We have more and more contributors involved in the development, and >> the number of contributors has reached 36! >> 6. I have created a github project to help schedule the development so >> that contributors could easily pick up issues they are interested in! >> >> cc Brian, I could update the progress to meeting notes if necessary. >> >> On Thu, Mar 28, 2024 at 6:50 AM Brian Olsen <bitsondata...@gmail.com> >> wrote: >> >>> Hey Iceberg Nation, >>> >>> Here are the meeting minutes from today's meeting. >>> >>> Transcription/Recording: https://youtu.be/9TRhgRq5bFk >>> Meeting Notes: >>> https://docs.google.com/document/d/1YuGhUdukLP5gGiqCbk0A5_Wifqe2CZWgOd3TbhY3UQg/edit#heading=h.686a3a6fup8m >>> >>> >>> New Issue Template for Spec and Design Proposals: Dan created a new >>> issue template and documentation to streamline design docs and spec >>> updates, improving visibility into discussions and future planning in the >>> Iceberg community. >>> Manifest Encryption Support: Manifest encryption support has been added, >>> further advancing Iceberg's encryption capabilities, including the ability >>> to handle encrypted manifests. >>> Function Pushdown in Spark: Anton enhanced function pushdown for >>> row-level commands in Spark, refining the handling of system function >>> pushdown for commands like merge and update. >>> Output Spec Selection in Rewrite Data Files: Himadri introduced a >>> feature allowing the selection of different output specs in rewrite data >>> files, enhancing the capability for data management and optimization. >>> >>> Releases: >>> 1.5.0: This version has been successfully released after numerous >>> release candidates. >>> 1.5.1: Potential upcoming release with discussions around schema >>> behaviors with time travel reads, Trino integration improvements, and an >>> extra HEAD request issue in S3 file IO. >>> 1.6.0: Planning to include an Arrow upgrade for CVE fixes. >>> PyIceberg 0.6.1: Focus on compatibility and bug fixes, such as >>> addressing the creation of version one tables with non-empty partition >>> specs and sort orders. >>> >>> Discussion: >>> Spec and Design Proposals: The implementation of a new issue template >>> and tagging system to track and manage proposals and discussions. >>> REST Protocol Evolution: Discussion about adding capabilities to the >>> REST protocol, ensuring client and server compatibility for new features. >>> Schema Behavior with Time Travel Reads: Addressing the issue with schema >>> behavior changes during time travel reads, aiming for consistency across >>> branches. >>> Flink Table Maintenance: A deep dive into Flink table maintenance >>> including strategies for compaction, orphan file removal, and snapshot >>> expiration. Discussion on handling streaming jobs, rolling manifest >>> writers, and the RewriteFileGroupPlanner. >>> Catalog Discussion: Deliberation on deprecating the DynamoDB catalog. >>> How to address challenges related to the Spark 3.3 and 3.5 merge into >>> Iceberg, specifically around dynamic partition pruning (DPP) issues. >>> >>
