Re: Key rotation in Iceberg data encryption

2021-03-25 Thread Gidon Gershinsky
want to rotate KEK can just use option 3.3. > > > > Best, > > Jack Ye > > > > *From: *Russell Spitzer > *Reply-To: *"dev@iceberg.apache.org" > *Date: *Thursday, March 25, 2021 at 08:33 > *To: *Iceberg Dev List > *Subject: *RE: [EXTERNAL] Key rotation

Re: Key rotation in Iceberg data encryption

2021-03-25 Thread Ye, Jack
t;dev@iceberg.apache.org" Date: Thursday, March 25, 2021 at 08:33 To: Iceberg Dev List Subject: RE: [EXTERNAL] Key rotation in Iceberg data encryption CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender an

Re: Key rotation in Iceberg data encryption

2021-03-25 Thread Russell Spitzer
I think you can treat the key rotation as a spark action like "RewriteManifestsAction" or something like that which creates a new Snapshot and new set of manifest files. If we want to be secure we would follow this up by immediately exporting and deleting previous snapshots and manifests. One probl

Key rotation in Iceberg data encryption

2021-03-25 Thread Gidon Gershinsky
Hi all, We're working with Jack on a design for encryption of Iceberg data tables, and got a question / decision point we'd like to bring to the community's attention. Might be a bit exotic, but is important, so we have to try this. Any input on this subject, or pointers to relevant contacts / sou