RE: Many S3V4AuthErrorRetryStrategy warn logs while reading/writing from S3

2021-09-24 Thread Hailu, Andreas
Thanks, Robert. // ah From: Robert Metzger Sent: Wednesday, September 22, 2021 1:49 PM To: Hailu, Andreas [Engineering] Cc: user@flink.apache.org Subject: Re: Many S3V4AuthErrorRetryStrategy warn logs while reading/writing from S3 Hey Andreas, This could be related too https://github.com

Re: Many S3V4AuthErrorRetryStrategy warn logs while reading/writing from S3

2021-09-22 Thread Robert Metzger
Hey Andreas, This could be related too https://github.com/apache/hadoop/pull/110/files#diff-0a2e55a2f79ea4079eb7b77b0dc3ee562b383076fa0ac168894d50c80a95131dR950 I guess in Flink this would be s3.endpoint: your-endpoint-hostname Where your-endpoint-hostname is a region-specific endpoint, which y

Many S3V4AuthErrorRetryStrategy warn logs while reading/writing from S3

2021-09-22 Thread Hailu, Andreas
Hi, When reading/writing to and from S3 using the flink-fs-s3-hadoop plugin on 1.11.2, we observe a lot of these WARN log statements in the logs: WARN S3V4AuthErrorRetryStrategy - Attempting to re-send the request to s3.amazonaws.com with AWS V4 authentication. To avoid this warning in the fu