Heads up: PR [1913] got approved. If there are no more comments, I'm going to merge tomorrow (EDT).
[1913] https://github.com/apache/polaris/pull/1913 Cheers, Dmitri. On Fri, Jun 27, 2025 at 8:47 PM Dmitri Bourlatchkov <di...@apache.org> wrote: > I added a separate STS endpoint property to > https://github.com/apache/polaris/pull/1913 > > Cheers, > Dmitri. > > On Wed, Jun 25, 2025 at 7:59 PM Dmitri Bourlatchkov <di...@apache.org> > wrote: > >> Thanks for the feedback, Adnan! >> >> I was initially thinking of deferring separating STS and S3 endpoints to >> a later PR, but given interest, I'll add it to the current PR to reduce the >> number of API changes over time. >> >> Cheers, >> Dmitri. >> >> On Wed, Jun 25, 2025 at 1:28 PM Adnan Hemani >> <adnan.hem...@snowflake.com.invalid> wrote: >> >>> +1 on overall idea >>> >>> Comment however on keeping the same endpoint for STS and S3 - there may >>> be >>> a customer use case (I have seen this before for similar use cases) where >>> the S3 bucket is not in the same region as the STS endpoint that has been >>> exposed to the Polaris server. In that case, you really need two separate >>> endpoints (example below). But I don't think this necessarily blocks this >>> idea from being merged today and then enriched later. >>> >>> Example: >>> * Polaris is deployed in us-west-1 in a EC2 instance that is within a >>> private subnet, which is given access only to STS' us-west-1 regional >>> endpoint through AWS configurations. >>> * User has S3 data in both us-west-1 and us-east-1. >>> * Under this proposal, user can now access data in us-west-1, but when >>> trying to access data in us-east-1, STS will not respond due to no >>> network >>> routing tables for the us-east-1 regional endpoint. >>> * Important thing to remember is that STS token generated at any regional >>> endpoint can be used at any other regional endpoint. So, in theory, user >>> could still use us-west-1 STS to generate credentials to use for S3 data >>> in >>> us-east-1 and it would've been ok - but this proposal does not allow for >>> this. >>> >>> Best, >>> Adnan Hemani >>> >>> On Tue, Jun 24, 2025 at 9:23 PM Dmitri Bourlatchkov <di...@apache.org> >>> wrote: >>> >>> > Hi All, >>> > >>> > I propose to add an `endpoint` optional parameter to >>> AwsStorageConfigInfo >>> > in PR [1913]. >>> > >>> > The main idea is to support non-AWS S3 implementations for [1530]. >>> > >>> > Existing classes related to supporting S3 Polaris are coded to the AWS >>> SDK, >>> > which supports setting STS endpoints. Therefore, it seems natural to >>> allow >>> > users to optionally define a specific endpoint for their catalogs. >>> > >>> > This change is backward-compatible with existing clients and deployed >>> > catalogs. >>> > >>> > When an endpoint is defined it will be used for both STS and S3 >>> requests >>> > inside Polaris and will be used to populate the "s3.endpoint" >>> properties in >>> > REST Catalog clients. >>> > >>> > Thoughts? >>> > >>> > [1530] https://github.com/apache/polaris/issues/1530 >>> > [1913] https://github.com/apache/polaris/pull/1913 >>> > >>> > Thanks, >>> > Dmitri. >>> > >>> >>
