Hi Harshith, Did you enable delete permission on S3 for running machines? Are you using IAM roles or access key id and secret access key combo?
-- Thanks, Amit On Mon, Oct 15, 2018 at 3:15 PM Kumar Bolar, Harshith <hk...@arity.com> wrote: > Hi all, > > > > We store Flink checkpoints in Amazon S3. Flink periodically sends out GET, > PUT, LIST, DELETE requests to S3, to store-clear checkpoints. From the > logs, we see that GET, PUT and LIST requests are successful but it throws > an AWS access denied error for DELETE request. > > > > Here’s a snippet of the logs for DELETE request – > > > > 2018-10-15 04:13:22,819 INFO > org.apache.flink.fs.s3presto.shaded.com.amazonaws.latency - > ServiceName=[Amazon S3], AWSErrorCode=[AccessDenied], StatusCode=[403], > ServiceEndpoint=[https://xxx-xxx-prod.s3.amazonaws.com], > Exception=[org.apache.flink.fs.s3presto.shaded.com.amazonaws.services.s3.model.AmazonS3Exception: > Access Denied (Service: Amazon S3; *Status Code: 403; Error Code: > AccessDenied;* Request ID: xxxxxxxxxxxxx), S3 Extended Request ID: > xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx], > *RequestType=[DeleteObjectRequest]*, AWSRequestID=[XXXXXXXXXXXXXXXXXX], > HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, > HttpClientPoolAvailableCount=1, RequestCount=1, Exception=1, > HttpClientPoolLeasedCount=0, ClientExecuteTime=[4.984], > HttpClientSendRequestTime=[0.029], HttpRequestTime=[4.84], > RequestSigningTime=[0.038], CredentialsRequestTime=[0.0, 0.0], > HttpClientReceiveResponseTime=[4.78] > > > > Is there some configuration that we’re forgetting that is preventing Flink > from sending DELETE requests to S3? > > > > I’d be happy to provide more information if needed. > > > > Thanks, > > Harshith > > > > >
