[
https://issues.apache.org/jira/browse/NIFI-16369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lewis J updated NIFI-16369:
---------------------------
Description:
{panel}
When using {{PutIcebergRecord}} with a REST catalog configured for OAuth 2.0
Client Credentials ({{{}rest.auth.type=oauth2{}}}), table load operations fail
unconditionally with an {{{}UnsupportedOperationException{}}}. The error occurs
because {{{}CredentialsRefreshRESTClient{}}}—which is only active when OAuth
2.0 client credentials are used—fails to forward interface default methods
added to Iceberg's {{RESTClient}} interface.
_(Note: Catalog configurations using Bearer token authentication are
unaffected, as {{CredentialsRefreshRESTClient}} is not installed for Bearer
auth.)_
h4. *Steps to Reproduce*
# Configure {{PutIcebergRecord}} to connect to an Iceberg REST catalog using
the OAuth 2.0 client-credentials strategy ({{{}rest.auth.type=oauth2{}}} with
Client ID and Client Secret).
# Trigger the processor to execute a write or load table operation.
h4. *Expected Result*
{{PutIcebergRecord}} successfully loads table metadata from the REST catalog
and processes incoming records.
h4. *Actual Result*
The processor throws an {{UnsupportedOperationException}} on every execution
trigger:
```java
{{java.lang.UnsupportedOperationException: Returning response headers is not
supported
at org.apache.iceberg.rest.RESTClient.get(RESTClient.java:131)
at
org.apache.iceberg.rest.RESTSessionCatalog.loadInternal(RESTSessionCatalog.java:438)
at
org.apache.nifi.processors.iceberg.PutIcebergRecord.getTable(PutIcebergRecord.java:223)}}
```
No configuration workaround avoids this when using OAuth 2.0 Client
Credentials, as {{RESTSessionCatalog.loadInternal}} calls the header-returning
{{get(...)}} overload unconditionally during table loads. Swapping to Bearer
token authentication works as a temporary workaround because it bypasses the
decorator entirely.
h4. *Root Cause Analysis*
{{CredentialsRefreshRESTClient}} is installed only when OAuth 2.0 client
credentials are configured. Its sole purpose is to wrap {{RESTClient}} to
intercept {{postForm}} calls, rewriting RFC 8693 token exchange requests into
standard client-credentials grants.
Upstream Iceberg's {{RESTClient}} interface defines opt-in default methods for
handling response headers, query parameters, and parser contexts. By design,
the default interface implementations throw {{UnsupportedOperationException}}
unless overridden by an implementing transport client.
Because {{CredentialsRefreshRESTClient}} directly implements {{RESTClient}} but
only overrides the base abstract methods, invocations of these interface
default methods silently bypass the wrapped delegate ({{{}HTTPClient{}}}) and
execute the throwing default methods on the interface.
The following five {{RESTClient}} default methods are unhandled and throw when
invoked through the decorator:
* {{get(String, Map, Class<T>, Map, Consumer<ErrorResponse>,
Consumer<Map<String,String>>)}} _(causes the table load failure)_
* {{get(String, Map, Class<T>, Map, Consumer<ErrorResponse>, ParserContext)}}
* {{post(String, RESTRequest, Class<T>, Map, Consumer<ErrorResponse>,
Consumer<Map<String,String>>)}}
* {{post(String, RESTRequest, Class<T>, Map, Consumer<ErrorResponse>,
Consumer<Map<String,String>>, ParserContext)}}
* {{delete(String, Map, Class<T>, Map, Consumer<ErrorResponse>)}}
h4. *Proposed Solutions*
Override and forward all five missing {{RESTClient}} default methods in
{{CredentialsRefreshRESTClient}} directly to the underlying {{restClient}}
delegate. Ensure {{withAuthSession}} continues to return a wrapped decorator
instance so session-bound operations maintain the token refresh behavior.
h5. *or possibly...*
Eliminate the {{CredentialsRefreshRESTClient}} decorator by connecting
{{RESTIcebergCatalog}} directly to NiFi's standard
{{OAuth2AccessTokenProvider}} Controller Service via Iceberg's {{AuthManager}}
SPI.
{panel}
was:
{panel}
When using {{PutIcebergRecord}} with a REST catalog configured for OAuth 2.0
Client Credentials ({{{}rest.auth.type=oauth2{}}}), table load operations fail
unconditionally with an {{{}UnsupportedOperationException{}}}. The error occurs
because {{{}CredentialsRefreshRESTClient{}}}—which is only active when OAuth
2.0 client credentials are used—fails to forward interface default methods
added to Iceberg's {{RESTClient}} interface.
_(Note: Catalog configurations using Bearer token authentication are
unaffected, as {{CredentialsRefreshRESTClient}} is not installed for Bearer
auth.)_
h4. *Steps to Reproduce*
# Configure {{PutIcebergRecord}} to connect to an Iceberg REST catalog using
the OAuth 2.0 client-credentials strategy ({{{}rest.auth.type=oauth2{}}} with
Client ID and Client Secret).
# Trigger the processor to execute a write or load table operation.
h4. *Expected Result*
{{PutIcebergRecord}} successfully loads table metadata from the REST catalog
and processes incoming records.
h4. *Actual Result*
The processor throws an {{UnsupportedOperationException}} on every execution
trigger:
```java
{{java.lang.UnsupportedOperationException: Returning response headers is not
supported
at org.apache.iceberg.rest.RESTClient.get(RESTClient.java:131)
at
org.apache.iceberg.rest.RESTSessionCatalog.loadInternal(RESTSessionCatalog.java:438)
at
org.apache.nifi.processors.iceberg.PutIcebergRecord.getTable(PutIcebergRecord.java:223)}}
```
No configuration workaround avoids this when using OAuth 2.0 Client
Credentials, as {{RESTSessionCatalog.loadInternal}} calls the header-returning
{{get(...)}} overload unconditionally during table loads. Swapping to Bearer
token authentication works as a temporary workaround because it bypasses the
decorator entirely.
h4. *Root Cause Analysis*
{{CredentialsRefreshRESTClient}} is installed only when OAuth 2.0 client
credentials are configured. Its sole purpose is to wrap {{RESTClient}} to
intercept {{postForm}} calls, rewriting RFC 8693 token exchange requests into
standard client-credentials grants.
Upstream Iceberg's {{RESTClient}} interface defines opt-in default methods for
handling response headers, query parameters, and parser contexts. By design,
the default interface implementations throw {{UnsupportedOperationException}}
unless overridden by an implementing transport client.
Because {{CredentialsRefreshRESTClient}} directly implements {{RESTClient}} but
only overrides the base abstract methods, invocations of these interface
default methods silently bypass the wrapped delegate ({{{}HTTPClient{}}}) and
execute the throwing default methods on the interface.
The following five {{RESTClient}} default methods are unhandled and throw when
invoked through the decorator:
* {{get(String, Map, Class<T>, Map, Consumer<ErrorResponse>,
Consumer<Map<String,String>>)}} _(causes the table load failure)_
* {{get(String, Map, Class<T>, Map, Consumer<ErrorResponse>, ParserContext)}}
* {{post(String, RESTRequest, Class<T>, Map, Consumer<ErrorResponse>,
Consumer<Map<String,String>>)}}
* {{post(String, RESTRequest, Class<T>, Map, Consumer<ErrorResponse>,
Consumer<Map<String,String>>, ParserContext)}}
* {{delete(String, Map, Class<T>, Map, Consumer<ErrorResponse>)}}
h4. *Proposed Solutions*
Override and forward all five missing {{RESTClient}} default methods in
{{CredentialsRefreshRESTClient}} directly to the underlying {{restClient}}
delegate. Ensure {{withAuthSession}} continues to return a wrapped decorator
instance so session-bound operations maintain the token refresh behavior.
h5. *or possibly...*
Eliminate the {{CredentialsRefreshRESTClient}} decorator by connecting
{{RESTIcebergCatalog}} directly to NiFi's standard
{{OAuth2AccessTokenProvider}} Controller Service via Iceberg's {{AuthManager}}
SPI.
*if the former...*
h4. *Regression Testing / Verification*
Add a unit test in {{nifi-iceberg-rest-catalog}} asserting that all declared
methods on {{RESTClient.class}} are explicitly overridden by
{{{}CredentialsRefreshRESTClient{}}}:
```java
{{@Testvoid testAllRESTClientMethodsOverridden() { for (Method method :
RESTClient.class.getMethods())
Unknown macro: \{ assertTrue(
Arrays.stream(CredentialsRefreshRESTClient.class.getDeclaredMethods())
.anyMatch(m -> m.getName().equals(method.getName()) &&
Arrays.equals(m.getParameterTypes(), method.getParameterTypes())),
"CredentialsRefreshRESTClient must explicitly forward RESTClient method}
}}}
```
{panel}
> RESTIcebergCatalog - CredentialsRefreshRESTClient throws
> UnsupportedOperationException on REST catalog table loads
> ------------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-16369
> URL: https://issues.apache.org/jira/browse/NIFI-16369
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 2.12.0, 2.13.0
> Environment: nifi-iceberg-rest-catalog
> Reporter: Lewis J
> Priority: Minor
>
> {panel}
> When using {{PutIcebergRecord}} with a REST catalog configured for OAuth 2.0
> Client Credentials ({{{}rest.auth.type=oauth2{}}}), table load operations
> fail unconditionally with an {{{}UnsupportedOperationException{}}}. The error
> occurs because {{{}CredentialsRefreshRESTClient{}}}—which is only active when
> OAuth 2.0 client credentials are used—fails to forward interface default
> methods added to Iceberg's {{RESTClient}} interface.
> _(Note: Catalog configurations using Bearer token authentication are
> unaffected, as {{CredentialsRefreshRESTClient}} is not installed for Bearer
> auth.)_
>
> h4. *Steps to Reproduce*
> # Configure {{PutIcebergRecord}} to connect to an Iceberg REST catalog using
> the OAuth 2.0 client-credentials strategy ({{{}rest.auth.type=oauth2{}}} with
> Client ID and Client Secret).
> # Trigger the processor to execute a write or load table operation.
> h4. *Expected Result*
> {{PutIcebergRecord}} successfully loads table metadata from the REST catalog
> and processes incoming records.
> h4. *Actual Result*
> The processor throws an {{UnsupportedOperationException}} on every execution
> trigger:
> ```java
> {{java.lang.UnsupportedOperationException: Returning response headers is not
> supported
> at org.apache.iceberg.rest.RESTClient.get(RESTClient.java:131)
> at
> org.apache.iceberg.rest.RESTSessionCatalog.loadInternal(RESTSessionCatalog.java:438)
> at
> org.apache.nifi.processors.iceberg.PutIcebergRecord.getTable(PutIcebergRecord.java:223)}}
> ```
> No configuration workaround avoids this when using OAuth 2.0 Client
> Credentials, as {{RESTSessionCatalog.loadInternal}} calls the
> header-returning {{get(...)}} overload unconditionally during table loads.
> Swapping to Bearer token authentication works as a temporary workaround
> because it bypasses the decorator entirely.
> h4. *Root Cause Analysis*
> {{CredentialsRefreshRESTClient}} is installed only when OAuth 2.0 client
> credentials are configured. Its sole purpose is to wrap {{RESTClient}} to
> intercept {{postForm}} calls, rewriting RFC 8693 token exchange requests into
> standard client-credentials grants.
> Upstream Iceberg's {{RESTClient}} interface defines opt-in default methods
> for handling response headers, query parameters, and parser contexts. By
> design, the default interface implementations throw
> {{UnsupportedOperationException}} unless overridden by an implementing
> transport client.
> Because {{CredentialsRefreshRESTClient}} directly implements {{RESTClient}}
> but only overrides the base abstract methods, invocations of these interface
> default methods silently bypass the wrapped delegate ({{{}HTTPClient{}}}) and
> execute the throwing default methods on the interface.
> The following five {{RESTClient}} default methods are unhandled and throw
> when invoked through the decorator:
> * {{get(String, Map, Class<T>, Map, Consumer<ErrorResponse>,
> Consumer<Map<String,String>>)}} _(causes the table load failure)_
> * {{get(String, Map, Class<T>, Map, Consumer<ErrorResponse>, ParserContext)}}
> * {{post(String, RESTRequest, Class<T>, Map, Consumer<ErrorResponse>,
> Consumer<Map<String,String>>)}}
> * {{post(String, RESTRequest, Class<T>, Map, Consumer<ErrorResponse>,
> Consumer<Map<String,String>>, ParserContext)}}
> * {{delete(String, Map, Class<T>, Map, Consumer<ErrorResponse>)}}
> h4. *Proposed Solutions*
> Override and forward all five missing {{RESTClient}} default methods in
> {{CredentialsRefreshRESTClient}} directly to the underlying {{restClient}}
> delegate. Ensure {{withAuthSession}} continues to return a wrapped decorator
> instance so session-bound operations maintain the token refresh behavior.
> h5. *or possibly...*
> Eliminate the {{CredentialsRefreshRESTClient}} decorator by connecting
> {{RESTIcebergCatalog}} directly to NiFi's standard
> {{OAuth2AccessTokenProvider}} Controller Service via Iceberg's
> {{AuthManager}} SPI.
> {panel}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
