mccheah commented on issue #20: Encryption in Data Files URL: https://github.com/apache/incubator-iceberg/issues/20#issuecomment-443786685 > So to write, the job needs to generate random file keys and have them encrypted. The KeyManager just needs the master key name and gives back the key version, the encrypted file key, and the unencrypted file key. To read, we need to decrypt the file key, so the job passes the master key name, the master key version, and the encrypted key to the KeyManager. This is different from the way we model key storage. We roughly follow this model: https://github.com/palantir/hadoop-crypto/blob/ac8d4474ee667121873bf0abf0674d83c78d8b90/crypto-keys/src/main/java/com/palantir/crypto2/keys/KeyStorageStrategy.java#L27, where for a given FileSystem the encryption key name is derived from the file path: https://github.com/palantir/hadoop-crypto/blob/6d9e05a1e667f150f7d98435e93a0dd6f3ea5c08/hadoop-crypto/src/main/java/com/palantir/crypto2/hadoop/EncryptedFileSystem.java#L115. Because we use hadoop-crypto right now for existing encryption we need to match the existing model. I was wondering how we could do this with the Iceberg API while still enabling the KMS model you propose.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
