Hi Yes, I concur: I think StorageLocationPreparer is interesting. It would be a pretty flexible way to verify the storage is in a state before table creation. It could be a no-op step when nothing is required by the storage.
Regards JB On Tue, Apr 21, 2026 at 6:39 AM Dmitri Bourlatchkov <[email protected]> wrote: > Yes, StorageLocationPreparer looks like a useful plugin point in general... > and certainly helps with NHS in GCS. > > Cheers, > Dmitri. > > On Mon, Apr 20, 2026 at 7:26 PM Sung Yun <[email protected]> wrote: > > > Thanks for driving this effort Varun, > > > > I think the new change introducing StorageLocationPreparer is an > > interesting idea. This would allow Polaris to check whether storage is > > ready before table creation, which is a feature gap today. I think it > will > > work well with managed storage solutions that require some setup before > > data is written to the location. > > > > Cheers, > > Sung > > > > On 2026/04/20 18:51:32 Dmitri Bourlatchkov wrote: > > > Hi All, > > > > > > Heads up: we discussed this PR in the Community Sync last week. Please > > > review actual changes in GH and provide feedback. > > > > > > Thanks, > > > Dmitri. > > > > > > On Thu, Apr 2, 2026 at 1:46 PM Dmitri Bourlatchkov <[email protected]> > > wrote: > > > > > > > HI Varun, > > > > > > > > Thanks for your contribution! I think it's quite timely as we have > some > > > > recent user interest in the community [4106]. > > > > > > > > Your design LGTM overall. I'll try to have a full review of the PR > > soon. > > > > > > > > Note for other reviewers: This feature involves a minor Polaris REST > > API > > > > change. > > > > > > > > Should the flag default to auto-detection in the future (e.g., > > > > checking if the bucket has HNS enabled at catalog creation time), > > or is > > > > an > > > > explicit opt-in flag the right long-term approach? > > > > > > > > > > > > Auto-detection is a nice to have feature, but I'm not sure it's worth > > the > > > > added complexity on the Polaris side. > > > > > > > > I believe (admin) users are generally aware of the HNS state of the > > bucket > > > > they use for a catalog, so having a HNS flag in the Storage > > Configuration > > > > is probably not burdensome. At least this is quite acceptable for the > > > > initial PR. Note that Azure has a similar flag in its Storage > > Configuration > > > > [3347]. > > > > > > > > This is just my personal opinion. If you feel like adding > > auto-detection > > > > in a follow-up PR, by all means please feel free to contribute that > > too. > > > > > > > > [3347] https://github.com/apache/polaris/pull/3347 > > > > > > > > [4106] https://github.com/apache/polaris/pull/4106 > > > > > > > > Cheers, > > > > Dmitri. > > > > > > > > On Thu, Apr 2, 2026 at 2:27 AM Varun Arya <[email protected]> > > wrote: > > > > > > > >> Hi Polaris community, > > > >> I'd like to start a discussion around a change I've been working on > > to add > > > >> support for GCS buckets with > > > >> https://cloud.google.com/storage/docs/hns-overview enabled. > > > >> > > > >> > > > >> > > > >> *Problem* > > > >> When a GCS bucket has HNS enabled, folders must be created > explicitly > > as > > > >> managed folders. The current credential vending logic only grants > > > >> `roles/storage.legacyObjectReader`, > > > >> `roles/storage.objectViewer`, and `roles/storage.legacyBucketWriter` > > which > > > >> is insufficient for HNS buckets. Operations that need to create > > folders > > > >> (e.g., Iceberg writing data/metadata) fail with 403 errors because > the > > > >> downscoped token lacks `storage.managedFolders.create` permission. > > > >> > > > >> > > > >> > > > >> *Proposed Solution* > > > >> I've added a new optional hierarchicalNamespace boolean flag to > > > >> GcpStorageConfigInfo in the management API. When set to true, the > > > >> credential vending logic generates an additional access boundary > rule > > > >> granting `roles/storage.folderAdmin` scoped to the specific write > > paths > > > >> using > > > >> > > > resource.name.startsWith('projects/_/buckets/<bucket>/managedFolders/<path>') > > > >> conditions. > > > >> > > > >> *Key design decisions*: > > > >> > > > >> 1. Opt-in flag: HNS support is not enabled by default. Users must > > > >> explicitly set hierarchicalNamespace: true in their catalog > storage > > > >> configuration. This avoids granting unnecessary permissions on > > non-HNS > > > >> buckets. > > > >> 2. Least-privilege scoping: `roles/storage.folderAdmin` is the > > > >> least-privileged predefined GCP role that includes > > > >> storage.managedFolders.create. The access boundary condition > > expression > > > >> limits scope to the specific write paths only. > > > >> 3. Write-path only: Folder management rules are only generated > for > > > >> write > > > >> locations. Read-only access does not get folderAdmin permissions. > > > >> 4. Multi-bucket support: The implementation handles cases where > > > >> metadata > > > >> and data reside in separate buckets, generating correctly scoped > > > >> folderAdmin rules for each. > > > >> > > > >> *Open Questions* > > > >> > > > >> 1. Should the flag default to auto-detection in the future (e.g., > > > >> checking if the bucket has HNS enabled at catalog creation time), > > or > > > >> is an > > > >> explicit opt-in flag the right long-term approach? > > > >> > > > >> PR link: Enable HNS support for GCS > > > >> <https://github.com/apache/polaris/pull/3996> > > > >> > > > >> Thanks, > > > >> Varun Arya > > > >> > > > > > > > > > >
