Andy Huang created HDDS-17087:
---------------------------------
Summary: S3 Gateway does not validate x-amz-content-sha256 for
UploadPart
Key: HDDS-17087
URL: https://issues.apache.org/jira/browse/HDDS-17087
Project: Apache Ozone
Issue Type: Bug
Components: S3
Reporter: Andy Huang
UploadPart accepts a well-formed x-amz-content-sha256 value that does not match
the uploaded part body. Both the regular and datastream multipart upload paths
are affected. Ordinary PutObject already rejects this mismatch with
XAmzContentSHA256Mismatch.
h3. Reproduction
# Initiate a multipart upload.
# Upload a part containing "Multipart Upload", with x-amz-content-sha256 set to
64 zero characters (a valid hexadecimal SHA-256 value, but not the digest of
this body).
# Repeat with datastream disabled and enabled.
Expected: UploadPart returns XAmzContentSHA256Mismatch before committing the
part; ListParts does not include the rejected part.
Actual: UploadPart succeeds and commits the part.
h3. Code observations
EndpointBase.getS3ChunkInputStreamInfo creates a SHA-256 MessageDigest for a
non-streaming payload hash. However, ObjectEndpoint.createMultipartKey and
ObjectEndpointStreaming.createMultipartKey do not compare that digest with the
supplied header before committing the part. These paths validate Content-MD5,
but omit the equivalent SHA-256 check.
h3. Validation
A temporary regression test in TestPartUpload used a matching digest for a
successful upload, then attempted a second part with a header containing 64
zero characters. The assertion expecting XAmzContentSHA256Mismatch failed
because no exception was thrown, with datastream both disabled and enabled.
This was an endpoint unit test using the existing OzoneClientStub; a
live-cluster reproduction has not yet been run.
h3. Proposed fix and tests
Validate the computed SHA-256 against the expected header before committing a
multipart part in both upload paths, reusing existing validation patterns. Add
tests for matching and mismatched digests, empty parts, and
uppercase/mixed-case hexadecimal headers. Verify rejected parts are absent from
ListParts and that a failed replacement preserves the previous part. Preserve
unsigned and streaming payload handling.
h3. Related work
* HDDS-13668 added single-chunk payload verification; this appears to be a
missing multipart path.
* HDDS-16660 concerns checksum values in trailers, which is separate from a
hexadecimal digest directly supplied in x-amz-content-sha256.
* Found while testing HDDS-17055 / https://github.com/apache/ozone/pull/11562.
The multipart comparison is already absent on that branch's master base.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
