[
https://issues.apache.org/jira/browse/HDDS-10688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tanvi Penumudy resolved HDDS-10688.
-----------------------------------
Resolution: Resolved
> S3 multipart upload failed for Chinese filename with s3 fuse clients
> --------------------------------------------------------------------
>
> Key: HDDS-10688
> URL: https://issues.apache.org/jira/browse/HDDS-10688
> Project: Apache Ozone
> Issue Type: Bug
> Components: S3, s3gateway
> Affects Versions: 1.4.0, 1.5.0
> Reporter: Soumitra Sulav
> Assignee: Tanvi Penumudy
> Priority: Critical
>
> * Issue is seen only with s3 fuse clients which internally do the copy via
> multipart upload mechanism. Basically, the client initiates an MPU, creates
> individual parts uploads, and finally runs the COMPLETE_MULTIPART_UPLOAD
> using individual parts. The issue is observed at the last layer where it is
> trying to merge the file and failing to do so.
> * This issue is only seen with Chinese (Non-English charset) characters.
> * Upload-part API misses the non-English character in the response even
> after setting proper encoding and locale variables.
> * List-Part API response is proper.
> Below are the repro steps :
> # Install goofys fuse client
> [https://github.com/kahing/goofys/releases/download/v0.24.0/goofys]
> # Mount the ozone s3 endpoint via goofys
> {code:java}
> goofys --debug_fuse --debug_s3 --endpoint
> http://<OzoneS3GHost>:<OzoneS3GPort> <BucketName> <LocalPath>
> {code}
> # Create a file of size > 5MB and a name containing Chinese characters.
> # Copy the file from the local filesystem to the mounted path.
> {code:java}
> cp 测试三.txt /mnt/test-goofys/
> cp: failed to close '/mnt/test-goofys/测试三.txt': Invalid argument
> {code}
> Error stacktrace
> {code:java}
> 2024-04-08 09:29:36,145 | INFO | S3GAudit | [email protected] |
> ip=10.129.77.95 | op=INIT_MULTIPART_UPLOAD {bucket=[buckettest],
> path=[测试三.txt], uploads=[]} | ret=SUCCESS |
> 2024-04-08 09:29:36,288 | INFO | S3GAudit | [email protected] |
> ip=10.129.77.95 | op=CREATE_MULTIPART_KEY {bucket=[buckettest],
> path=[测试三.txt],
> uploadId=[669be17b-6c05-4066-9398-13a3586c65b1-112234894206109441],
> partNumber=[1]} | ret=SUCCESS |
> 2024-04-08 09:29:36,432 | INFO | S3GAudit | [email protected] |
> ip=10.129.77.95 | op=CREATE_MULTIPART_KEY {bucket=[buckettest],
> path=[测试三.txt],
> uploadId=[669be17b-6c05-4066-9398-13a3586c65b1-112234894206109441],
> partNumber=[2]} | ret=SUCCESS |
> 2024-04-08 09:29:36,455 | ERROR | S3GAudit | [email protected] |
> ip=10.129.77.95 | op=COMPLETE_MULTIPART_UPLOAD {bucket=[buckettest],
> path=[测试三.txt],
> uploadId=[669be17b-6c05-4066-9398-13a3586c65b1-112234894206109441]} |
> ret=FAILURE | INVALID_PART org.apache.hadoop.ozone.om.exceptions.OMException:
> Complete Multipart Upload Failed: volume: s3v bucket: buckettest key:
> 测试三.txt. Provided Part info is { /s3v/buckettest/
> .txt-669be17b-6c05-4066-9398-13a3586c65b1-112234894206109441-1, 1}, whereas
> OM has partName
> /s3v/buckettest/测试三.txt-669be17b-6c05-4066-9398-13a3586c65b1-112234894206109441-1
> at
> org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:728)
> at
> org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.completeMultipartUpload(OzoneManagerProtocolClientSideTranslatorPB.java:1587)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
