Bablu Raul created HDDS-17658:
---------------------------------
Summary: S3 CreateMultipartUpload Returns HTTP 500 InternalError
When the Key Matches an Existing Directory in an FSO Bucket
Key: HDDS-17658
URL: https://issues.apache.org/jira/browse/HDDS-17658
Project: Apache Ozone
Issue Type: Bug
Reporter: Bablu Raul
When {{CreateMultipartUpload}} is called in a {{FILE_SYSTEM_OPTIMIZED (FSO)}}
bucket with an object key that matches an existing directory, the S3 Gateway
returns HTTP 500 InternalError instead of an appropriate 4xx client error.
This issue occurs with both {{dir1}} and {{dir1/}} when the directory {{dir1/}}
already exists in the bucket.
The Ozone Manager correctly rejects the request with {{OMException}} result
code {{{}NOT_A_FILE{}}}:
{{NOT_A_FILE: Can not write to directory: dir1}}
However, the S3 Gateway does not translate this exception into an appropriate
S3 error for the {{CreateMultipartUpload}} operation. As a result, the
exception is propagated as a generic HTTP 500 InternalError.
h2. Steps to Reproduce
*1. Create an FSO bucket:*
{{export BKT=b4-repro-$(date +%H%M%S)
ozone sh bucket create /s3v/$BKT --layout FILE_SYSTEM_OPTIMIZED
ozone sh bucket info /s3v/$BKT | grep -i bucketLayout}}
Verify that the bucket layout is {{{}FILE_SYSTEM_OPTIMIZED{}}}.
*2. Create a directory through the S3 API:*
{{aws s3api put-object --bucket "$BKT" --key dir1/}}
Verify the directory:
{{ozone sh key list /s3v/$BKT}}
Expected entry:
{{"name" : "dir1/", "dataSize" : 0, "file" : false}}
*3. Initiate a multipart upload using the directory name without a trailing
slash:*
{{aws s3api create-multipart-upload --bucket "$BKT" --key dir1}}
Actual result:
{{An error occurred (InternalError) when calling the CreateMultipartUpload
operation (reached max retries: 2): We encountered an internal error. Please
try again.}}
*4. Repeat using the directory name with a trailing slash:*
{{aws s3api create-multipart-upload --bucket "$BKT" --key dir1/}}
Actual result: The same HTTP 500 InternalError response.
*5. Compare with PutObject using the same key:*
{{echo hello > /tmp/b4.txt
aws s3api put-object --bucket "$BKT" --key dir1 --body /tmp/b4.txt}}
Actual result: HTTP 400 InvalidRequest with a file-system-paths validation
message.
h2. Actual Result
* {{CreateMultipartUpload}} returns HTTP 500 InternalError when the key
matches an existing directory.
* The behavior occurs for both {{dir1}} and {{{}dir1/{}}}.
* The AWS CLI retries the request, resulting in three requests in total.
* The S3 Gateway logs the exception as an internal server error.
h2. Expected Result
The S3 Gateway should handle the OM {{NOT_A_FILE}} exception and return an
appropriate 4xx S3 error.
Preferably, it should return HTTP 400 InvalidRequest with the same
file-system-paths validation message returned by {{{}PutObject{}}}.
* No HTTP 500 InternalError.
* No unnecessary client retries.
* No ERROR-level InternalError logging for this expected client-side
validation failure.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
