Hey Andreas, This could be related too https://github.com/apache/hadoop/pull/110/files#diff-0a2e55a2f79ea4079eb7b77b0dc3ee562b383076fa0ac168894d50c80a95131dR950
I guess in Flink this would be s3.endpoint: your-endpoint-hostname Where your-endpoint-hostname is a region-specific endpoint, which you can probably look up from the S3 docs. On Wed, Sep 22, 2021 at 7:07 PM Hailu, Andreas <andreas.ha...@gs.com> wrote: > Hi, > > > > When reading/writing to and from S3 using the flink-fs-s3-hadoop plugin on > 1.11.2, we observe a lot of these WARN log statements in the logs: > > > > *WARN S3V4AuthErrorRetryStrategy - Attempting to re-send the request to > s3.amazonaws.com <http://s3.amazonaws.com> with AWS V4 authentication. To > avoid this warning in the future, please use region-specific endpoint to > access buckets located in regions that require V4 signing.* > > > > The applications complete successfully which is great, but I’m not sure > what the root of the error is and I’m hesitant to silence it through our > logging configurations. I saw something that looks similar here[1]. Is > there a way for us to similarly have Flink’s AWS S3 client to use V4 > strategy to begin with? > > > > [1] > https://stackoverflow.com/questions/39513518/aws-emr-writing-to-kms-encrypted-s3-parquet-files > > > > ____________ > > > > *Andreas Hailu* > > *Data Lake Engineering *| Goldman Sachs & Co. > > > > ------------------------------ > > Your Personal Data: We may collect and process information about you that > may be subject to data protection laws. For more information about how we > use and disclose your personal data, how we protect your information, our > legal basis to use your information, your rights and who you can contact, > please refer to: www.gs.com/privacy-notices >
