Hi all, I'd like to propose adding HashiCorp Vault as a first-class KMS implementation in Apache Iceberg's encryption framework.
*Background* Iceberg's encryption spec supports pluggable KMS backends, but the set of officially supported providers is still limited. HashiCorp Vault is one of the most widely adopted secrets management and encryption-as-a-service platforms in the industry. The Vault repository <https://github.com/hashicorp/vault> has over 35k GitHub stars, and it's a common choice in enterprises that prefer self-hosted or cloud-agnostic key management. *Motivation* I recently received a request to support Vault-encrypted Iceberg tables in Trino. Rather than implementing this in the Trino repository alone, I believe the right place for this is upstream in Iceberg itself. Adding it here means: 1. Any engine benefits immediately - Spark, Flink, Trino, and others can all interoperate with Vault-encrypted tables without each needing their own implementation. 2. No duplicated effort - a single, well-tested integration in Iceberg avoids the maintenance burden of per-engine implementations diverging over time. 3. Consistency across the ecosystem - users get the same configuration interface and behavior regardless of which engine they use. *Proposal* I've opened PR #16075 <https://github.com/apache/iceberg/pull/16075> as a starting point. It introduces: - A new iceberg-hashicorp module containing the Vault KMS client implementation - A new hashicorp value for the kms-type configuration property *Questions for the Community* - Is there any objection to supporting Vault in the Iceberg repository? - Are there any concerns about introducing a dependency on the Vault client library? Happy to discuss the design further. Thanks for taking a look. BR, Yuya Ebihara
