Shuo Huang created HDDS-16672:
---------------------------------
Summary: S3 Gateway corrupts Unicode user metadata values
Key: HDDS-16672
URL: https://issues.apache.org/jira/browse/HDDS-16672
Project: Apache Ozone
Issue Type: Sub-task
Reporter: Shuo Huang
Assignee: Shuo Huang
h3. Problem
S3 Gateway corrupts non-ASCII characters in user-defined object metadata.
For example, an object uploaded with:
{{x-amz-meta-meta1: Hello Worldé}}
returns:
{{x-amz-meta-meta1: Hello Worldé}}
AWS S3 supports Unicode characters in user-defined metadata values.
h3. Reproduction
Start an Ozone cluster and configure ceph/s3-tests to use its S3 endpoint, then
run:
{{S3TEST_CONF=<path-to-s3tests.conf> python -m pytest -v
s3tests/functional/test_s3.py::test_object_set_get_unicode_metadata}}
h3. Actual result
The test fails with:
{{AssertionError: assert 'Hello Worldé' == 'Hello Worldé'}}
h3. Expected result
The metadata value returned by GetObject should match the uploaded value:
{{Hello Worldé}}
h3. Root cause
The UTF-8 header bytes are exposed to S3 Gateway as a byte-preserving Latin-1
string. {{EndpointBase.getCustomMetadataFromHeaders()}} stores that string
without decoding it back to UTF-8.
h3. References
* [AWS S3 user-defined
metadata|https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html]
* [ceph/s3-tests
reproduction|https://github.com/ceph/s3-tests/blob/fb8b73092bb1dd8db829f1205a9e52e73bf9a232/s3tests/functional/test_s3.py#L1885-L1899]
* [Ozone S3 compatibility report|https://ozone.s3.peterxcli.dev/]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
