Re: [DISCUSS] Changes for row-level deletes

2020-05-06 Thread Junjie Chen
Hi Ryan Besides the reading and merging of delete files, can we talk a bit about write side of delete files? For example, generate delete files in a spark action, the metadata column support, the service to transfer equality delete files to position delete files etc.. On Wed, May 6, 2020 at 1:34

Re: [DISCUSS] Changes for row-level deletes

2020-05-06 Thread Anton Okolnychyi
We won’t have to rewrite V1 metadata when migrating to V2. The format is backward compatible and we can read V1 manifests just fine in V2. For example, V1 metadata will not have have sequence number and V2 would interpret that as sequence number = 0. The only thing we need to prohibit is V1 writ

Fwd: [RESULT] [VOTE] Release Apache Iceberg 0.8.0-incubating RC2

2020-05-06 Thread Ryan Blue
The release vote on the IPMC list has passed! I'll start the process to publish the release. -- Forwarded message - From: Ryan Blue Date: Wed, May 6, 2020 at 1:06 PM Subject: [RESULT] [VOTE] Release Apache Iceberg 0.8.0-incubating RC2 To: Owen O'Malley Cc: Apache Incubator Wit

Re: [DISCUSS] Changes for row-level deletes

2020-05-06 Thread Gautam
My 2 cents : > * Merge manifest_entry and data_file? ... -1 .. keeping the difference between v1 and v2 metadata to a minimum would be my preference by keeping manifest_entries the same way in both v1 and v2. People using either flows would want to modify and contribute and shouldn't have