Soumitra Sulav created HDDS-10688:
-------------------------------------
Summary: S3 multipart upload failed for Chinese filename with s3
fuse clients
Key: HDDS-10688
URL: https://issues.apache.org/jira/browse/HDDS-10688
Project: Apache Ozone
Issue Type: Bug
Components: S3, s3gateway
Affects Versions: 1.4.0, 1.5.0
Reporter: Soumitra Sulav
* Issue is seen only with s3 fuse clients which internally do the copy via
multipart upload mechanism. Basically, the client initiates an MPU, creates
individual parts uploads, and finally runs the COMPLETE_MULTIPART_UPLOAD using
individual parts. The issue is observed at the last layer where it is trying to
merge the file and failing to do so.
* This issue is only seen with Chinese (Non-English charset) characters.
* Upload-part API misses the non-English character in the response even after
setting proper encoding and locale variables.
* List-Part API response is proper.
Below are the repro steps :
# Install goofys fuse client
[https://github.com/kahing/goofys/releases/download/v0.24.0/goofys]
# Mount the ozone s3 endpoint via goofys
{code:java}
goofys --debug_fuse --debug_s3 --endpoint http://<OzoneS3GHost>:<OzoneS3GPort>
<BucketName> <LocalPath>
{code}
# Create a file of size > 5MB and a name containing Chinese characters.
# Copy the file from the local filesystem to the mounted path.
{code:java}
cp 测试三.txt /mnt/test-goofys/
cp: failed to close '/mnt/test-goofys/测试三.txt': Invalid argument
{code}
Error stacktrace
{code:java}
2024-04-08 09:29:36,145 | INFO | S3GAudit | [email protected] |
ip=10.129.77.95 | op=INIT_MULTIPART_UPLOAD {bucket=[buckettest],
path=[测试三.txt], uploads=[]} | ret=SUCCESS |
2024-04-08 09:29:36,288 | INFO | S3GAudit | [email protected] |
ip=10.129.77.95 | op=CREATE_MULTIPART_KEY {bucket=[buckettest], path=[测试三.txt],
uploadId=[669be17b-6c05-4066-9398-13a3586c65b1-112234894206109441],
partNumber=[1]} | ret=SUCCESS |
2024-04-08 09:29:36,432 | INFO | S3GAudit | [email protected] |
ip=10.129.77.95 | op=CREATE_MULTIPART_KEY {bucket=[buckettest], path=[测试三.txt],
uploadId=[669be17b-6c05-4066-9398-13a3586c65b1-112234894206109441],
partNumber=[2]} | ret=SUCCESS |
2024-04-08 09:29:36,455 | ERROR | S3GAudit | [email protected] |
ip=10.129.77.95 | op=COMPLETE_MULTIPART_UPLOAD {bucket=[buckettest],
path=[测试三.txt],
uploadId=[669be17b-6c05-4066-9398-13a3586c65b1-112234894206109441]} |
ret=FAILURE | INVALID_PART org.apache.hadoop.ozone.om.exceptions.OMException:
Complete Multipart Upload Failed: volume: s3v bucket: buckettest key: 测试三.txt.
Provided Part info is { /s3v/buckettest/
.txt-669be17b-6c05-4066-9398-13a3586c65b1-112234894206109441-1, 1}, whereas OM
has partName
/s3v/buckettest/测试三.txt-669be17b-6c05-4066-9398-13a3586c65b1-112234894206109441-1
at
org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:728)
at
org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.completeMultipartUpload(OzoneManagerProtocolClientSideTranslatorPB.java:1587)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
