[
https://issues.apache.org/jira/browse/HDDS-16672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-16672:
----------------------------------
Labels: pull-request-available (was: )
> S3 Gateway corrupts Unicode user metadata values
> ------------------------------------------------
>
> Key: HDDS-16672
> URL: https://issues.apache.org/jira/browse/HDDS-16672
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Shuo Huang
> Assignee: Shuo Huang
> Priority: Major
> Labels: pull-request-available
>
> h3. Problem
> S3 Gateway corrupts non-ASCII characters in user-defined object metadata.
> For example, an object uploaded with:
> {{x-amz-meta-meta1: Hello Worldé}}
> returns:
> {{x-amz-meta-meta1: Hello Worldé}}
> AWS S3 supports Unicode characters in user-defined metadata values.
> h3. Reproduction
> Start an Ozone cluster and configure ceph/s3-tests to use its S3 endpoint,
> then run:
> {{S3TEST_CONF=<path-to-s3tests.conf> python -m pytest -v
> s3tests/functional/test_s3.py::test_object_set_get_unicode_metadata}}
> h3. Actual result
> The test fails with:
> {{AssertionError: assert 'Hello Worldé' == 'Hello Worldé'}}
> h3. Expected result
> The metadata value returned by GetObject should match the uploaded value:
> {{Hello Worldé}}
> h3. Root cause
> The UTF-8 header bytes are exposed to S3 Gateway as a byte-preserving Latin-1
> string. {{EndpointBase.getCustomMetadataFromHeaders()}} stores that string
> without decoding it back to UTF-8.
> h3. References
> * [AWS S3 user-defined
> metadata|https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html]
> * [ceph/s3-tests
> reproduction|https://github.com/ceph/s3-tests/blob/fb8b73092bb1dd8db829f1205a9e52e73bf9a232/s3tests/functional/test_s3.py#L1885-L1899]
> * [Ozone S3 compatibility report|https://ozone.s3.peterxcli.dev/]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
