github-advanced-security[bot] commented on code in PR #18891:
URL: https://github.com/apache/druid/pull/18891#discussion_r2916020539
##########
extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3Utils.java:
##########
@@ -328,23 +371,41 @@
public static void deleteBucketKeys(
ServerSideEncryptingAmazonS3 s3Client,
String bucket,
- List<DeleteObjectsRequest.KeyVersion> keysToDelete,
+ List<ObjectIdentifier> keysToDelete,
int retries
)
throws Exception
{
if (keysToDelete != null && log.isDebugEnabled()) {
List<String> keys = keysToDelete.stream()
-
.map(DeleteObjectsRequest.KeyVersion::getKey)
+ .map(ObjectIdentifier::key)
.collect(Collectors.toList());
log.debug("Deleting keys from bucket: [%s], keys: [%s]", bucket, keys);
}
- DeleteObjectsRequest deleteRequest = new
DeleteObjectsRequest(bucket).withKeys(keysToDelete);
- S3Utils.retryS3Operation(() -> {
- s3Client.deleteObjects(deleteRequest);
- return null;
- }, retries);
- log.info("Deleted %d files", keysToDelete.size());
+ List<ObjectIdentifier> remaining = keysToDelete;
+ List<S3Error> lastErrors = null;
+ for (int attempt = 0; attempt <= retries; attempt++) {
+ DeleteObjectsRequest deleteRequest = DeleteObjectsRequest.builder()
+ .bucket(bucket)
+ .delete(Delete.builder().objects(remaining).build())
+ .build();
+ DeleteObjectsResponse response = retryS3Operation(() ->
s3Client.deleteObjects(deleteRequest));
+ if (!response.hasErrors()) {
+ log.info("Deleted %d files", keysToDelete.size());
Review Comment:
## Dereferenced variable may be null
Variable [keysToDelete](1) may be null at this access as suggested by
[this](2) null guard.
[Show more
details](https://github.com/apache/druid/security/code-scanning/10875)
##########
extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3DataSegmentKiller.java:
##########
@@ -147,65 +153,74 @@
private boolean deleteKeysForBucket(
ServerSideEncryptingAmazonS3 s3Client,
String s3Bucket,
- List<DeleteObjectsRequest.KeyVersion> keysToDelete
+ List<ObjectIdentifier> keysToDelete
)
{
boolean hadException = false;
- DeleteObjectsRequest deleteObjectsRequest = new
DeleteObjectsRequest(s3Bucket);
- deleteObjectsRequest.setQuiet(true);
- List<List<DeleteObjectsRequest.KeyVersion>> keysChunks = Lists.partition(
+ List<List<ObjectIdentifier>> keysChunks = Lists.partition(
keysToDelete,
MAX_MULTI_OBJECT_DELETE_SIZE
);
- for (List<DeleteObjectsRequest.KeyVersion> chunkOfKeys : keysChunks) {
- List<String> keysToDeleteStrings = chunkOfKeys.stream().map(
-
DeleteObjectsRequest.KeyVersion::getKey).collect(Collectors.toList());
+ for (List<ObjectIdentifier> chunkOfKeys : keysChunks) {
try {
- deleteObjectsRequest.setKeys(chunkOfKeys);
log.info(
"Deleting the following segment files from S3 bucket[%s]: [%s]",
s3Bucket,
- keysToDeleteStrings
- );
- S3Utils.retryS3Operation(
- () -> {
- s3Client.deleteObjects(deleteObjectsRequest);
- return null;
- },
- 3
+
chunkOfKeys.stream().map(ObjectIdentifier::key).collect(Collectors.toList())
);
- }
- catch (MultiObjectDeleteException e) {
- hadException = true;
- Map<String, List<String>> errorToKeys = new HashMap<>();
- for (MultiObjectDeleteException.DeleteError error : e.getErrors()) {
- errorToKeys.computeIfAbsent(StringUtils.format(
- MULTI_OBJECT_DELETE_EXEPTION_ERROR_FORMAT,
- error.getMessage(),
- error.getCode()
- ), k -> new ArrayList<>()).add(error.getKey());
+ List<ObjectIdentifier> remaining = chunkOfKeys;
+ for (int attempt = 0; attempt <= NUM_RETRIES; attempt++) {
Review Comment:
## Useless comparison test
Test is always true, because of [this condition](1).
Test is always true, because of [this condition](2).
[Show more
details](https://github.com/apache/druid/security/code-scanning/10876)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
