[
https://issues.apache.org/jira/browse/HDDS-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Huang updated HDDS-17087:
------------------------------
Description:
UploadPart accepts a well-formed x-amz-content-sha256 value that does not match
the uploaded part body. Both the regular and datastream multipart upload paths
are affected. Ordinary PutObject already rejects this mismatch with
XAmzContentSHA256Mismatch.
h3. Reproduction
# Initiate a multipart upload.
# Upload a part containing "Multipart Upload", with x-amz-content-sha256 set to
64 zero characters (a valid hexadecimal SHA-256 value, but not the digest of
this body).
# Repeat with datastream disabled and enabled.
Expected: UploadPart returns XAmzContentSHA256Mismatch before committing the
part; ListParts does not include the rejected part.
Actual: UploadPart succeeds and commits the part.
h3. Code observations
EndpointBase.getS3ChunkInputStreamInfo creates a SHA-256 MessageDigest for a
non-streaming payload hash. However, ObjectEndpoint.createMultipartKey and
ObjectEndpointStreaming.createMultipartKey do not compare that digest with the
supplied header before committing the part. These paths validate Content-MD5,
but omit the equivalent SHA-256 check.
h3. Validation
A temporary regression test in TestPartUpload used a matching digest for a
successful upload, then attempted a second part with a header containing 64
zero characters. The assertion expecting XAmzContentSHA256Mismatch failed
because no exception was thrown, with datastream both disabled and enabled.
This was an endpoint unit test using the existing OzoneClientStub; a
live-cluster reproduction has not yet been run.
h3. Proposed fix and tests
Validate the computed SHA-256 against the expected header before committing a
multipart part in both upload paths, reusing existing validation patterns. Add
tests for matching and mismatched digests, empty parts, and
uppercase/mixed-case hexadecimal headers. Verify rejected parts are absent from
ListParts and that a failed replacement preserves the previous part. Preserve
unsigned and streaming payload handling.
was:
UploadPart accepts a well-formed x-amz-content-sha256 value that does not match
the uploaded part body. Both the regular and datastream multipart upload paths
are affected. Ordinary PutObject already rejects this mismatch with
XAmzContentSHA256Mismatch.
h3. Reproduction
# Initiate a multipart upload.
# Upload a part containing "Multipart Upload", with x-amz-content-sha256 set to
64 zero characters (a valid hexadecimal SHA-256 value, but not the digest of
this body).
# Repeat with datastream disabled and enabled.
Expected: UploadPart returns XAmzContentSHA256Mismatch before committing the
part; ListParts does not include the rejected part.
Actual: UploadPart succeeds and commits the part.
h3. Code observations
EndpointBase.getS3ChunkInputStreamInfo creates a SHA-256 MessageDigest for a
non-streaming payload hash. However, ObjectEndpoint.createMultipartKey and
ObjectEndpointStreaming.createMultipartKey do not compare that digest with the
supplied header before committing the part. These paths validate Content-MD5,
but omit the equivalent SHA-256 check.
h3. Validation
A temporary regression test in TestPartUpload used a matching digest for a
successful upload, then attempted a second part with a header containing 64
zero characters. The assertion expecting XAmzContentSHA256Mismatch failed
because no exception was thrown, with datastream both disabled and enabled.
This was an endpoint unit test using the existing OzoneClientStub; a
live-cluster reproduction has not yet been run.
h3. Proposed fix and tests
Validate the computed SHA-256 against the expected header before committing a
multipart part in both upload paths, reusing existing validation patterns. Add
tests for matching and mismatched digests, empty parts, and
uppercase/mixed-case hexadecimal headers. Verify rejected parts are absent from
ListParts and that a failed replacement preserves the previous part. Preserve
unsigned and streaming payload handling.
h3. Related work
* HDDS-13668 added single-chunk payload verification; this appears to be a
missing multipart path.
* HDDS-16660 concerns checksum values in trailers, which is separate from a
hexadecimal digest directly supplied in x-amz-content-sha256.
* Found while testing HDDS-17055 / https://github.com/apache/ozone/pull/11562.
The multipart comparison is already absent on that branch's master base.
> S3 Gateway does not validate x-amz-content-sha256 for UploadPart
> ----------------------------------------------------------------
>
> Key: HDDS-17087
> URL: https://issues.apache.org/jira/browse/HDDS-17087
> Project: Apache Ozone
> Issue Type: Bug
> Components: S3
> Reporter: Andy Huang
> Priority: Major
>
> UploadPart accepts a well-formed x-amz-content-sha256 value that does not
> match the uploaded part body. Both the regular and datastream multipart
> upload paths are affected. Ordinary PutObject already rejects this mismatch
> with XAmzContentSHA256Mismatch.
> h3. Reproduction
> # Initiate a multipart upload.
> # Upload a part containing "Multipart Upload", with x-amz-content-sha256 set
> to 64 zero characters (a valid hexadecimal SHA-256 value, but not the digest
> of this body).
> # Repeat with datastream disabled and enabled.
> Expected: UploadPart returns XAmzContentSHA256Mismatch before committing the
> part; ListParts does not include the rejected part.
> Actual: UploadPart succeeds and commits the part.
> h3. Code observations
> EndpointBase.getS3ChunkInputStreamInfo creates a SHA-256 MessageDigest for a
> non-streaming payload hash. However, ObjectEndpoint.createMultipartKey and
> ObjectEndpointStreaming.createMultipartKey do not compare that digest with
> the supplied header before committing the part. These paths validate
> Content-MD5, but omit the equivalent SHA-256 check.
> h3. Validation
> A temporary regression test in TestPartUpload used a matching digest for a
> successful upload, then attempted a second part with a header containing 64
> zero characters. The assertion expecting XAmzContentSHA256Mismatch failed
> because no exception was thrown, with datastream both disabled and enabled.
> This was an endpoint unit test using the existing OzoneClientStub; a
> live-cluster reproduction has not yet been run.
> h3. Proposed fix and tests
> Validate the computed SHA-256 against the expected header before committing a
> multipart part in both upload paths, reusing existing validation patterns.
> Add tests for matching and mismatched digests, empty parts, and
> uppercase/mixed-case hexadecimal headers. Verify rejected parts are absent
> from ListParts and that a failed replacement preserves the previous part.
> Preserve unsigned and streaming payload handling.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
