[
https://issues.apache.org/jira/browse/NIFI-14588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17952974#comment-17952974
]
Peter Turcsanyi commented on NIFI-14588:
----------------------------------------
[~timsloane] Thanks for reporting the issue!
In NiFi 1.x with, there was a
[logic|https://github.com/apache/nifi/blob/7aaaba2c8331dc310d9ec192926ff72f24a138ab/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java#L342-L355]
to parse the Endpoint Override URL and set the Region based on that. In case
of regular endpoints, the AWS library can extract the Region and override the
configured one. For VPCE and other custom endpoints, the NiFi code tries to
parse the URL if possible, otherwise falls back to the configured Region.
Either way, the retrieved Region is used for the service and signing regions as
well.
There were 2 changes in NiFi 2.x:
* the custom endpoint parsing was removed and only the AWS library is used for
parsing the Endpoint Override URL => the newer version of the library may be
able to handle all types of endpoints but it needs to be checked
* the configured Region is statically set in the
[EndpointConfiguration|https://github.com/apache/nifi/blob/203bde19f46b6a9347d0f7cec2b0b71dcdcb2d76/nifi-extension-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSCredentialsProviderProcessor.java#L259]
object as the signing region => this is the root cause of the current issue
because there is a mismatch between the service region (parsed from the URL)
and the signing region (set from the processor's Region property) which leads
to the "authorization header is malformed" error
Passing null (meaning default) as the signing region seems to work. Will check
it in more detail.
> S3 processors fail with an Endpoint Override URL
> ------------------------------------------------
>
> Key: NIFI-14588
> URL: https://issues.apache.org/jira/browse/NIFI-14588
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 2.3.0
> Environment: RHEL9 running on a cluster of three t3.medium EC2s on
> AWS.
> Reporter: Tim Sloane
> Priority: Minor
> Labels: s3
>
> When I try to use ListS3, FetchS3Object, or PutS3Object with Endpoint
> Override URL set, NiFi fails with:
> {{AmazonS3Exception: The authorization header is malformed: the region
> 'us-west-2' is wrong; expecting '{_}<Endpoint Override URL region>{_}'}}
> I just migrated from NiFi 1.28.0 to 2.3, and this problem didn't exist in
> 1.28.0 (or any versions going back to 1.9 for that matter). The 2.0
> documentation still indicates Endpoint Override URL should cause the selected
> Region to be ignored, but that seems to not be the case.
> I do have a workaround where I set Region to "Use 's3.region' Attribute" and
> set s3.region in a preceding UpdateAttribute.
> My guess is this is a bug introduced in 2.x.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
