Re: Meeting Minutes from 10/20 Iceberg Sync

Thanks for the detailed report ! One more thing: We now have made a lot of progress in integrating Alibaba Cloud (https://www.aliyun.com/), Please see https://github.com/apache/iceberg/projects/21 (Thanks @xingbowu - https://github.com/xingbowu). On Thu, Oct 21, 2021 at 11:30 PM Sam Redai wrote

Re: Iceberg Delete Compaction Interface Design

Had some offline discussions on Slack and WeChat. For Russell's point, we are reconfirming with related people on Slack, and will post updates once we have an agreement. Regarding point 6, for Flink CDC the data file flushed to disk might be associated with position deletes, but after the flush a

Re: Iceberg Delete Compaction Interface Design

I think I understood the Rewrite strategy discussion a little differently Binpack Strategy and SortStrategy each get a new flag which lets you pick files based on their number of delete files. So basically you can set a variety of parameters, small files, large files, files with deletes etc ... A

Re: Iceberg Delete Compaction Interface Design

Thanks to everyone who came to the meeting. Here is the full meeting recording I made: https://drive.google.com/file/d/1yuBFlNn9nkMlH9TIut2H8CXmJGLd18Sa/view?usp=sharing Here are some key takeaways: 1. we generally agreed upon the division of compactions into Rewrite, Convert and Merge. 2. Merg

Re: Iceberg 0.12.1 Patch Release - Call for Bug Fixes and Patches

Thank you everybody for the additional PRs brought up so far. I’ve volunteered to be release manager, so will be doing my best to go through and ensure these are prioritized for consideration (if some are truly new features they might need to wait for 0.13.0, but as I’m just the release manager th

Re: Iceberg 0.12.1 Patch Release - Call for Bug Fixes and Patches

Just to make this clean https://github.com/apache/iceberg/pull/3338 fixes the issue caused by https://github.com/apache/iceberg/pull/2565 . The fix will make Catalogs.loadCatalog consistent with Catalogs.

Meeting Minutes from 10/20 Iceberg Sync

Good Morning Everyone, Here are the minutes from our Iceberg Sync that took place on October 20th, 9am-10am PT. Please remember that anyone can join the discussion so feel free to share the Iceberg-Sync google group with anyone who is seeking an invite. A

Re: Iceberg 0.12.1 Patch Release - Call for Bug Fixes and Patches

I would like to have this in 0.12.1: https://github.com/apache/iceberg/pull/3338 This breaks Hive queries, if no catalog is set, but this still needs to be reviewed before merge. Thanks, Peter On Thu, 21 Oct 2021, 07:12 Rajarshi Sarkar, wrote: > Hope this can get in: https://github.com/apache