Hi Alex, I checked our code further. We have an internal implementation for RESTClient which needs an update after your change to place the token in DefaultAuthSession and not pass through initHeaders anymore. The existing code assumes the token coming from initHeaders and we need to make the change to get the token from the headers of DefaultAuthSession.
With that, +1 (non-binding) Thanks, Aihua On Thu, Feb 27, 2025 at 9:26 AM Alex Dutra <alex.du...@dremio.com.invalid> wrote: > Hi Aihua Xu, > > I reviewed your PR but without further details I do not agree with your > change, and I am unable to reproduce the issue. Besides, we have unit > tests that cover this extensively. > > Could you please provide a simple reproducer or a test case? > > Thanks! > > Alex > > On Thu, Feb 27, 2025 at 5:06 PM Aihua Xu <aihu...@gmail.com> wrote: > >> I'm running within our integration tests. I'm able to trace and find the >> breaking change: https://github.com/apache/iceberg/pull/11992. >> >> Seems we need to make the following changes to pass the token: >> https://github.com/apache/iceberg/pull/12415/files. Alex, can you help >> take a look at this? >> >> if (hasCredential) { >> authResponse = >> OAuth2Util.fetchToken( >> initClient, initHeaders, credential, scope, oauth2ServerUri, >> optionalOAuthParams); >> Map<String, String> authHeaders = >> RESTUtil.merge(initHeaders, >> OAuth2Util.authHeaders(authResponse.token())); >> config = fetchConfig(initClient, authHeaders, props); >> } else { >> authResponse = null; >> Map<String, String> authHeaders = >> RESTUtil.merge(initHeaders, OAuth2Util.authHeaders(initToken)); >> config = fetchConfig(initClient, authHeaders, props); >> } >> >> >> On Thu, Feb 27, 2025 at 12:46 AM Alex Dutra <alex.du...@dremio.com.invalid> >> wrote: >> >>> Hi Aihua, >>> >>> I just tested 1.8.1 with Polaris OSS and I am not seeing anything >>> different. Can you share your setup? >>> >>> Below is my Spark setup. >>> >>> Thanks, >>> >>> Alex >>> >>> ./gradlew run >>> >>> token=$(curl -s http://localhost:8181/api/catalog/v1/oauth/tokens \ >>> --user root:<secret> \ >>> -d grant_type=client_credentials \ >>> -d scope=PRINCIPAL_ROLE:ALL | sed -n >>> 's/.*"access_token":"\([^"]*\)".*/\1/p') >>> >>> curl -s -H "Authorization: Bearer ${token}" \ >>> -H 'Accept: application/json' \ >>> -H 'Content-Type: application/json' \ >>> http://localhost:8181/api/management/v1/catalogs \ >>> -d '{ >>> "catalog": { >>> "name": "polaris_demo", >>> "type": "INTERNAL", >>> "readOnly": false, >>> "properties": { >>> "default-base-location": "file:///tmp/polaris/" >>> }, >>> "storageConfigInfo": { >>> "storageType": "FILE", >>> "allowedLocations": [ >>> "file:///tmp" >>> ] >>> } >>> } >>> }' >>> >>> >>> spark-sql \ >>> --packages org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.8.1 \ >>> --conf >>> spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions >>> \ >>> --conf >>> spark.sql.catalog.polaris=org.apache.iceberg.spark.SparkCatalog \ >>> --conf >>> spark.sql.catalog.polaris.catalog-impl=org.apache.iceberg.rest.RESTCatalog \ >>> --conf spark.sql.catalog.polaris.uri= >>> http://127.0.0.1:8181/api/catalog \ >>> --conf spark.sql.catalog.polaris.credential=root:<secret> \ >>> --conf spark.sql.catalog.polaris.scope=PRINCIPAL_ROLE:ALL \ >>> --conf spark.sql.catalog.polaris.warehouse=polaris_demo >>> >>> spark-sql (default)> create namespace polaris.foo; >>> 25/02/27 09:43:24 WARN RESTSessionCatalog: Iceberg REST client is >>> missing the OAuth2 server URI configuration and defaults to >>> http://127.0.0.1:8181/api/catalog/v1/oauth/tokens. This automatic >>> fallback will be removed in a future Iceberg release.It is recommended to >>> configure the OAuth2 endpoint using the 'oauth2-server-uri' property to be >>> prepared. This warning will disappear if the OAuth2 endpoint is explicitly >>> configured. See https://github.com/apache/iceberg/issues/10537 >>> Time taken: 0.688 seconds >>> spark-sql (default)> >>> >>> >>> On Thu, Feb 27, 2025 at 12:22 AM Aihua Xu <aihu...@gmail.com> wrote: >>> >>>> I tested 1.8.1 RC with Snowflake build. I'm seeing the following (I'm >>>> not seeing that in 1.7.x). >>>> >>>> "exception": "java.io.IOException: *Authorization header is missing*\n\tat >>>> org.apache.polaris.service.dropwizard.auth.PolarisPrincipalAuthenticator.filter(PolarisPrincipalAuthenticator.java:43)\n\tat >>>> org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:108)\n\tat >>>> org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:44)\n\tat >>>> org.glassfish.jersey.process.internal.Stages.process(Stages.java:173)\n\tat >>>> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:248). >>>> >>>> Does anyone know which change could introduce this? >>>> >>>> On Wed, Feb 26, 2025 at 5:16 AM Péter Váry <peter.vary.apa...@gmail.com> >>>> wrote: >>>> >>>>> +1 >>>>> >>>>> checked the signatures, checksums >>>>> build and run some tests >>>>> >>>>> Amogh Jahagirdar <2am...@gmail.com> ezt írta (időpont: 2025. febr. >>>>> 26., Sze, 6:11): >>>>> >>>>>> +1 (binding) >>>>>> >>>>>> Verified signatures, checksum, RAT checks. >>>>>> Ran build and test with JDK17 >>>>>> >>>>>> Thanks, >>>>>> Amogh Jahagirdar >>>>>> >>>>>> On Wed, Feb 26, 2025 at 2:30 AM Honah J. <hon...@apache.org> wrote: >>>>>> >>>>>>> +1 (binding) >>>>>>> >>>>>>> - Checked signatures and checksum >>>>>>> - Checked license >>>>>>> - Full Build and Test >>>>>>> >>>>>>> Best regards, >>>>>>> Honah >>>>>>> >>>>>>> On Tue, Feb 25, 2025 at 10:52 AM Russell Spitzer < >>>>>>> russell.spit...@gmail.com> wrote: >>>>>>> >>>>>>>> +1 >>>>>>>> Checked Sigs and Checksum >>>>>>>> Ran Rat >>>>>>>> Ran full build/test >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Feb 25, 2025 at 11:30 AM Driesprong, Fokko >>>>>>>> <fo...@driesprong.frl> wrote: >>>>>>>> >>>>>>>>> +1 (binding) >>>>>>>>> >>>>>>>>> - Checked signatures and checksum >>>>>>>>> - Checked licenses >>>>>>>>> - Spotchecked NOTICE/LICENSE >>>>>>>>> >>>>>>>>> Kind regards, >>>>>>>>> Fokko >>>>>>>>> >>>>>>>>> Op di 25 feb 2025 om 16:56 schreef Kevin Liu < >>>>>>>>> kevinjq...@apache.org>: >>>>>>>>> >>>>>>>>>> +1 (non-binding) >>>>>>>>>> >>>>>>>>>> I followed "How to Verify a Release" >>>>>>>>>> <https://iceberg.apache.org/how-to-release/#how-to-verify-a-release> >>>>>>>>>> . >>>>>>>>>> Checked out artifact from SVN, >>>>>>>>>> ``` >>>>>>>>>> svn checkout >>>>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.8.1-rc1/ >>>>>>>>>> . >>>>>>>>>> ``` >>>>>>>>>> >>>>>>>>>> Verified >>>>>>>>>> * Signature Good >>>>>>>>>> * Checksum Ok >>>>>>>>>> * RAT check passed. 1 unrelated error message >>>>>>>>>> ``` >>>>>>>>>> ERROR: Ignored 0 lines in your exclusion files as comments or >>>>>>>>>> empty lines. >>>>>>>>>> ``` >>>>>>>>>> * Build + test passed, running on Java 17.0.6 (openjdk 17.0.6 >>>>>>>>>> 2023-01-17 LTS) on M1 >>>>>>>>>> * Ran a few examples on Spark >>>>>>>>>> * Ran pyiceberg integration tests, >>>>>>>>>> https://github.com/kevinjqliu/iceberg-python/pull/11 >>>>>>>>>> >>>>>>>>>> I ran the tests both with and without the docker daemon. Without >>>>>>>>>> docker, a few tests failed in `iceberg-aws`, `iceberg-azure`, and >>>>>>>>>> `iceberg-kafka-connect`. There's already an issue to track this at >>>>>>>>>> https://github.com/apache/iceberg/issues/12236. >>>>>>>>>> I'm also continuing to see the flakey test for `iceberg-core`'s >>>>>>>>>> `testConcurrentFastAppends` test. I believe this is a local issue >>>>>>>>>> with my >>>>>>>>>> machine. >>>>>>>>>> >>>>>>>>>> Thanks for running the release, Eduard! >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> Kevin Liu >>>>>>>>>> >>>>>>>>>> On Tue, Feb 25, 2025 at 4:23 AM Jean-Baptiste Onofré < >>>>>>>>>> j...@nanthrax.net> wrote: >>>>>>>>>> >>>>>>>>>>> +1 (non binding) >>>>>>>>>>> >>>>>>>>>>> - Hash and checksum are good >>>>>>>>>>> - LICENSE and NOTICE are OK in different distributed artifacts >>>>>>>>>>> (source >>>>>>>>>>> distribution, aws bundle, etc) >>>>>>>>>>> - ASF header present in all expected files >>>>>>>>>>> - No binary file found in the source distribution >>>>>>>>>>> - Did quick smoke tests >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Regards >>>>>>>>>>> JB >>>>>>>>>>> >>>>>>>>>>> On Mon, Feb 24, 2025 at 1:46 PM Eduard Tudenhoefner >>>>>>>>>>> <etudenhoef...@gmail.com> wrote: >>>>>>>>>>> > >>>>>>>>>>> > Hi Everyone, >>>>>>>>>>> > >>>>>>>>>>> > I propose that we release the following RC as the official >>>>>>>>>>> Apache Iceberg 1.8.1 release. >>>>>>>>>>> > >>>>>>>>>>> > The commit ID is 9ce0fcf0af7becf25ad9fc996c3bad2afdcfd33d >>>>>>>>>>> > * This corresponds to the tag: apache-iceberg-1.8.1-rc1 >>>>>>>>>>> > * >>>>>>>>>>> https://github.com/apache/iceberg/commits/apache-iceberg-1.8.1-rc1 >>>>>>>>>>> > * >>>>>>>>>>> https://github.com/apache/iceberg/tree/9ce0fcf0af7becf25ad9fc996c3bad2afdcfd33d >>>>>>>>>>> > >>>>>>>>>>> > The release tarball, signature, and checksums are here: >>>>>>>>>>> > * >>>>>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.8.1-rc1 >>>>>>>>>>> > >>>>>>>>>>> > You can find the KEYS file here: >>>>>>>>>>> > * https://downloads.apache.org/iceberg/KEYS >>>>>>>>>>> > >>>>>>>>>>> > Convenience binary artifacts are staged on Nexus. The Maven >>>>>>>>>>> repository URL is: >>>>>>>>>>> > * >>>>>>>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1184/ >>>>>>>>>>> > >>>>>>>>>>> > Please download, verify, and test. >>>>>>>>>>> > >>>>>>>>>>> > Please vote in the next 72 hours. >>>>>>>>>>> > >>>>>>>>>>> > [ ] +1 Release this as Apache Iceberg 1.8.1 >>>>>>>>>>> > [ ] +0 >>>>>>>>>>> > [ ] -1 Do not release this because... >>>>>>>>>>> > >>>>>>>>>>> > Only PMC members have binding votes, but other community >>>>>>>>>>> members are encouraged to cast >>>>>>>>>>> > non-binding votes. This vote will pass if there are 3 binding >>>>>>>>>>> +1 votes and more binding >>>>>>>>>>> > +1 votes than -1 votes. >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>>
