Ritesh Shukla created HDDS-17630:
------------------------------------
Summary: S3G: a part upload with an empty or missing uploadId
overwrites the object instead of failing
Key: HDDS-17630
URL: https://issues.apache.org/jira/browse/HDDS-17630
Project: Apache Ozone
Issue Type: Bug
Components: S3
Reporter: Ritesh Shukla
Assignee: Ritesh Shukla
h3. Problem
An S3 {{PUT}} on an object key means one of two things:
* {{PUT /bucket/key?partNumber=N&uploadId=<id>}} uploads one part of a
multipart upload (UploadPart). Nothing under the key changes until the upload
is completed.
* {{PUT /bucket/key}} writes the object (PutObject).
When {{uploadId}} is empty, or {{partNumber}} comes with no {{uploadId}} at
all, the S3 Gateway handles the request as a PutObject, or as a CopyObject when
{{x-amz-copy-source}} is set. The body (or the copy source) replaces the
committed object, and the client gets {{200 OK}}, just as for a part upload:
{code}
PUT /bucket/key body "original object" -> 200, key
holds "original object"
PUT /bucket/key?partNumber=1&uploadId= body "part 1" -> 200, key
now holds "part 1"
{code}
AWS S3 refuses each of these requests and leaves the object alone (tested
against AWS S3 on 2026-10-31). The Ozone column was run in the S3 Gateway
unit-test harness on master:
|| Request || AWS S3 || S3 Gateway on master ||
| {{PUT ?partNumber=1&uploadId=}} | 400 InvalidArgument, object kept | 200,
object replaced by the body |
| {{PUT ?partNumber=1&uploadId}} (no {{=}}) | 400 InvalidArgument, object kept
| same as above: the gateway's JAX-RS layer decodes a bare {{uploadId}} to the
same empty value (checked with Jersey's query decoder) |
| {{PUT ?partNumber=1}} (no {{uploadId}}) | 400 InvalidArgument, object kept |
200, object replaced by the body |
| {{PUT ?partNumber=1&uploadId=}} with {{x-amz-copy-source}} (UploadPartCopy) |
400 InvalidArgument, object kept | 200, object replaced by a full copy of the
source; {{x-amz-copy-source-range}} is ignored |
| {{PUT ?uploadId=}} (no {{partNumber}}) | 405 MethodNotAllowed, object kept |
200, object replaced by the body |
S3's InvalidArgument message is "This operation does not accept partNumber
without uploadId". With an empty body, the gateway truncates the object to 0
bytes.
The other handlers that read {{uploadId}} already answer as S3 does, except
DELETE:
* {{POST ?uploadId=}} (CompleteMultipartUpload) and {{GET ?uploadId=}}
(ListParts): S3 and the gateway both answer 404 NoSuchUpload. On the gateway
side this was run against the test client; that OM refuses an empty upload ID
was read from code.
* {{DELETE ?uploadId=}}: S3 answers 404 NoSuchUpload; the gateway deletes the
object. That is HDDS-17599.
h3. Impact
* Only a caller who may already write the key can do this. The operation that
actually runs is authorized as itself: PutObject, or GetObject on the source
and PutObject on the destination for the copy form. No permission boundary is
crossed (read from code).
* What is lost: the object's data (run), and with it the object's metadata and
tags (read from code). The response is a success, so the client cannot tell. A
client that uploads several parts under the same empty ID ends up with only the
last part as the object (inferred).
* How a client gets here: an application bug that passes an empty upload ID,
for example after losing the CreateMultipartUpload response. Stock clients send
it as-is. The AWS SDK for Java v2 (2.55.6) and v1 (1.12.788), and the AWS CLI
v2 (2.35.10), all put {{uploadId=}} on the wire for an empty ID, including for
UploadPartCopy; both SDKs refuse a null ID before sending (run against a local
capture server).
h3. Cause
{{ObjectEndpoint#handlePutRequest}} treats a request as a part upload only when
{{uploadID != null && !uploadID.equals("")}} (ObjectEndpoint.java:214 on
master). Every other request becomes a PutObject, or a CopyObject when
{{x-amz-copy-source}} is set (ObjectEndpoint.java:236-243). That path never
reads {{partNumber}}.
The empty-string test dates from the first UploadPart code (HDDS-902). The
parameter was then declared with {{@DefaultValue("")}}, so a missing
{{uploadId}} arrived as an empty string, and comparing with {{""}} was how
"absent" was written. HDDS-14209 switched to reading query parameters directly,
where a missing {{uploadId}} is {{null}}, but kept the comparison. So a present
but empty {{uploadId}} still counts as absent.
h3. Proposed fix
Answer as S3 does (tested against AWS S3 on 2026-10-31), before any OM call:
* {{partNumber}} present and {{uploadId}} missing or empty: 400
{{InvalidArgument}} with S3's message, "This operation does not accept
partNumber without uploadId". This also covers UploadPartCopy.
* {{uploadId}} present but empty, and no {{partNumber}}: 405
{{MethodNotAllowed}}. {{S3ErrorTable}} gains a {{METHOD_NOT_ALLOWED}} entry
with S3's code and message.
* Everything else is unchanged: a non-empty {{uploadId}} is still a part
upload, and a PUT with neither parameter is still a PutObject.
Both checks go at the top of {{handlePutRequest}}. A refused request is audited
and counted as a failed PutObject, and the key is never opened. The part-upload
branch then only needs {{uploadID != null}}.
The obvious one-line fix, dropping {{!uploadID.equals("")}} so the empty ID
reaches OM, is worse. OM would answer 404 NoSuchUpload, not S3's 400. {{PUT
?partNumber=1}} with no {{uploadId}} would still overwrite the object. And
UploadPartCopy would read the source key before failing. HDDS-17599 answers
DELETE with NoSuchUpload because that is what S3 does for DELETE; S3 answers
PUT differently, so this fix does too.
h3. Testing
* Unit, {{TestPartUpload}}: one parameterized case for each of
{{?partNumber=1&uploadId=}}, {{?partNumber=1}} and {{?uploadId=}}. Each puts a
key, sends the PUT, checks S3's error code and message, and checks that the
key's content and size are unchanged. It runs with and without data stream. On
master all six runs fail: the PUT succeeds and the key is replaced.
* Unit, {{TestMultipartUploadWithCopy}}: the same for UploadPartCopy with
{{?partNumber=1&uploadId=}}. On master the destination is replaced by the
source.
* {{EndpointTestUtils#put}} stops adding {{partNumber=0}} to plain PUTs. A real
PutObject carries no {{partNumber}}, and the new check would refuse it.
* All s3gateway unit tests pass with the change.
* Integration (to add with the fix): a case in {{AbstractS3SDKV2Tests}} calls
{{uploadPart}} and {{uploadPartCopy}} with {{uploadId("")}}, which the SDK
sends as {{uploadId=}}. It expects 400 InvalidArgument and the same ETag from
{{headObject}} afterwards.
----
🤖 Generated with [Claude Code|https://claude.com/claude-code]
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
