[
https://issues.apache.org/jira/browse/HDDS-17639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18136413#comment-18136413
]
Chi-Hsuan Huang commented on HDDS-17639:
----------------------------------------
While reviewing HDDS-17645, I noticed a negative {{part-number-marker}} has the
same problem as the empty one listed here. A value like {{-1}} parses fine but
fails the {{partNumberMarker >= 0}} precondition in {{RpcClient.listParts}}, so
ListParts returns HTTP 500 (read, not run).
It would be great if we can cover it together with that item.
> S3G: nine small S3 compatibility and robustness gaps found while testing
> request handling against AWS S3
> --------------------------------------------------------------------------------------------------------
>
> Key: HDDS-17639
> URL: https://issues.apache.org/jira/browse/HDDS-17639
> Project: Apache Ozone
> Issue Type: Improvement
> Components: S3
> Reporter: Ritesh Shukla
> Assignee: Ritesh Shukla
> Priority: Minor
>
> h3. Summary
> While testing how the S3 Gateway reads request bodies and query parameters
> against AWS S3 (HDDS-17600 and its follow-ups HDDS-17629 to HDDS-17632), we
> found nine smaller problems. They are collected here so they can be fixed
> together or split out. "Run" means measured, on a cluster or in the gateway's
> unit-test harness; "read" means found in the code only. "Tested on S3" means
> the request was sent to AWS S3 on 2026-10-31. Line numbers are for master
> {{ef63bacef7}}.
> h3. Requests answered differently from S3, or with a server error
> * *A {{Part}} with no {{ETag}} gets HTTP 500.* A CompleteMultipartUpload body
> with {{<Part><PartNumber>1</PartNumber></Part>}} passes a null ETag to the
> protobuf builder's {{setPartName}} in
> {{OmMultipartUploadCompleteList.getPartsList}}, which throws a
> NullPointerException; the client gets Jetty's HTML error page with status 500
> (run, mini cluster). S3's answer was not tested; 400 {{MalformedXML}} is
> likely.
> * *An empty {{Key}} is sent to OM.* For a DeleteObjects entry whose key is
> empty, AWS S3 answers 400 {{UserKeyMustBeSpecified}} (tested on S3 with
> {{<Key><a/></Key>}}, which S3 reads as an empty key). The gateway sends the
> empty name to OM and answers 200 with an empty {{Deleted}} entry (run). The
> gateway has no {{UserKeyMustBeSpecified}} error code.
> * *Elements S3 does not define are ignored.* AWS S3 refuses an element its
> schema does not define, even an empty one, with 400 {{MalformedXML}}:
> {{<Foo/>}} or {{<Foo>x</Foo>}} inside {{Delete}}, {{Object}} or {{Part}}
> (tested on S3). The gateway ignores them and acts on the rest of the body
> (run). A fix must keep accepting elements S3 defines but the gateway does not
> model, such as the checksum elements of a {{Part}} and the {{URI}} of a group
> grantee in an ACL body.
> * *An empty {{part-number-marker}} gets HTTP 500.* {{GET
> /bucket/key?uploadId=<id>&part-number-marker=}} (ListParts) calls
> {{Integer.parseInt("")}} ({{EndpointBase.java:868-874}}, from
> {{MultipartKeyHandler.java:63}}). The NumberFormatException is not an S3
> error, so the client gets HTTP 500 (run, mini cluster). S3's answer was not
> tested; 400 {{InvalidArgument}} is likely.
> * *{{HEAD ?uploadId=}} and {{GET ?partNumber=1&uploadId=}}.* AWS S3 answers
> both with 405 {{MethodNotAllowed}} (tested on S3). The gateway answers the
> first as HeadObject (200) and the second as ListParts (404 {{NoSuchUpload}})
> (run). Both are reads; nothing is changed.
> * *An ACL grantee read as {{CanonicalUser}} with no {{ID}} gets a server
> error.* A PutBucketAcl grantee whose type attribute binds the XML Schema
> instance namespace to a prefix other than {{xsi}}, such as {{i:type="Group"}}
> or {{i:type="AmazonCustomerByEmail"}}, is read as a CanonicalUser grantee
> without an ID. {{OzoneAcl.validateNameAndType}} then throws a
> NullPointerException ({{OzoneAcl.java:145}}, reached from
> {{S3Acl.java:224}}), and the request fails with a server error (run, unit
> harness). A grantee written with {{xsi:type="CanonicalUser"}} and no {{ID}}
> reaches the same line (read). S3's answer to these bodies was not tested.
> h3. Inside the gateway
> * *A failed part upload is counted twice in the failure metrics.* One failed
> UploadPart adds 1 to the part-upload failure counter and 1 to the create-key
> failure counter; one failed UploadPartCopy adds 1 to the copy failure counter
> and 1 to the create-key failure counter (run, unit harness).
> {{createMultipartKey}} counts its failure ({{ObjectEndpoint.java:1115-1118}}
> and {{1131-1134}}) and rethrows, and the outer catch in {{handlePutRequest}}
> counts it again ({{ObjectEndpoint.java:343-348}}).
> * *Lifecycle builds a new JAXB context for every request.*
> {{BucketLifecycleHandler}} creates a new
> {{PutBucketLifecycleConfigurationUnmarshaller}} for each
> PutBucketLifecycleConfiguration ({{BucketLifecycleHandler.java:137}}), and
> its constructor builds a {{JAXBContext}} (read). The other request-body
> readers are built once and reused.
> * *The unit-test client stub is looser than OM.* {{OzoneBucketStub}} keeps a
> deleted key's bytes; its {{listParts}}, {{completeMultipartUpload}} and
> {{abortMultipartUpload}} check only that some upload exists on the key, not
> the upload ID; {{createMultipartKey}} accepts any part number, including 0
> and numbers above 10,000; {{listParts}} hides part 0; and
> {{completeMultipartUpload}} answers {{InvalidPartOrder}} for part numbers at
> or below 0, where OM answers {{InvalidPart}} (read). Gateway unit tests can
> therefore pass against the stub for requests a real cluster refuses. Test
> code only.
> h3. Suggested handling
> Most items are small and independent: the missing ETag, the empty key, the
> empty part-number-marker, the HEAD and GET answers, the metrics, the
> per-request JAXB context and the stub. Two need more care. The ACL item
> touches the same code as the ACL parts of HDDS-17632 and is best done with
> them. Unknown elements need a list of the elements S3 defines for each
> request, so that refusing junk does not refuse fields newer SDKs send.
> ----
> 🤖 Generated with [Claude Code|https://claude.com/claude-code]
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
