Re: Plans for the future iceberg 0.11.0 release

OpenInx, thanks a lot for kicking off the discussion. Looks like my previous reply didn't reach the mailing list. > flink source based on the new FLIP-27 interface Yes, we shall target 0.11.0 release for the FLIP-27 flink source. I have updated the issue [1] with the following scopes. - Suppo

[VOTE] Release Apache Iceberg 0.10.0 RC2

Hi everyone, I propose the following RC to be released as official Apache Iceberg 0.10.0 release. The commit id is 37f21b72fb55503e6e40b1555b7ea1af61dfdfc7 * This corresponds to the tag: apache-iceberg-0.10.0-rc2 * https://github.com/apache/iceberg/commits/apache-iceberg-0.10.0-rc2

Re: [VOTE] Release Apache Iceberg 0.10.0 RC2

Here is the link to steps we normally use to validate a release candidate: https://lists.apache.org/thread.html/rd5e6b1656ac80252a9a7d473b36b6227da91d07d86d4ba4bee10df66%40%3Cdev.iceberg.apache.org%3E

Re: [VOTE] Release Apache Iceberg 0.10.0 RC2

+1 (non-binding) Downloaded and ran build with Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.7+8-LTS, mixed mode). All tests passed :) On Fri, Oct 30, 2020 at 4:05 PM Anton Okolnychyi wrote: > Here is the link to steps we normally use to validate a release candidate: > > https://lists.apach

Migrating plain parquet tables to iceberg

I’m looking to migrate a partitioned parquet table to use iceberg. The issue I’ve run into is that the column order for the data varies wildly, which isn’t a problem for us normally (we just set mergeSchemas=true when reading), but presents a problem with iceberg because the iceberg.schema field

Sync notes for 21 October

Hi everyone, I’ve updated the sync doc with notes from the sync last week. Sorry for the delay getting those written up. Highlights: - Congratulations to new committers Jingsong Lee and OpenInx! - Flink and MR read paths apply row-level deletes (thanks Junjie!) - Hive supports 3.x and c

Re: Migrating plain parquet tables to iceberg

For existing tables that use name-based column resolution, you can add a name-to-id mapping that is applied when reading files with no field IDs. There is a utility to generate the name mapping from an existing schema (using the current names) and then you just need to store that in a table propert

Dev list moderation

Hi everyone, I just found some unmoderated emails to this list that didn't make it to my inbox. If you've sent a question recently that didn't get a response, it may be that it was flagged for moderation and we missed it. I've updated my rules and gone through recent moderation requests and let th