Hi Aihua Xu, I reviewed your PR but without further details I do not agree with your change, and I am unable to reproduce the issue. Besides, we have unit tests that cover this extensively.
Could you please provide a simple reproducer or a test case? Thanks! Alex On Thu, Feb 27, 2025 at 5:06 PM Aihua Xu <aihu...@gmail.com> wrote: > I'm running within our integration tests. I'm able to trace and find the > breaking change: https://github.com/apache/iceberg/pull/11992. > > Seems we need to make the following changes to pass the token: > https://github.com/apache/iceberg/pull/12415/files. Alex, can you help > take a look at this? > > if (hasCredential) { > authResponse = > OAuth2Util.fetchToken( > initClient, initHeaders, credential, scope, oauth2ServerUri, > optionalOAuthParams); > Map<String, String> authHeaders = > RESTUtil.merge(initHeaders, > OAuth2Util.authHeaders(authResponse.token())); > config = fetchConfig(initClient, authHeaders, props); > } else { > authResponse = null; > Map<String, String> authHeaders = > RESTUtil.merge(initHeaders, OAuth2Util.authHeaders(initToken)); > config = fetchConfig(initClient, authHeaders, props); > } > > > On Thu, Feb 27, 2025 at 12:46 AM Alex Dutra <alex.du...@dremio.com.invalid> > wrote: > >> Hi Aihua, >> >> I just tested 1.8.1 with Polaris OSS and I am not seeing anything >> different. Can you share your setup? >> >> Below is my Spark setup. >> >> Thanks, >> >> Alex >> >> ./gradlew run >> >> token=$(curl -s http://localhost:8181/api/catalog/v1/oauth/tokens \ >> --user root:<secret> \ >> -d grant_type=client_credentials \ >> -d scope=PRINCIPAL_ROLE:ALL | sed -n >> 's/.*"access_token":"\([^"]*\)".*/\1/p') >> >> curl -s -H "Authorization: Bearer ${token}" \ >> -H 'Accept: application/json' \ >> -H 'Content-Type: application/json' \ >> http://localhost:8181/api/management/v1/catalogs \ >> -d '{ >> "catalog": { >> "name": "polaris_demo", >> "type": "INTERNAL", >> "readOnly": false, >> "properties": { >> "default-base-location": "file:///tmp/polaris/" >> }, >> "storageConfigInfo": { >> "storageType": "FILE", >> "allowedLocations": [ >> "file:///tmp" >> ] >> } >> } >> }' >> >> >> spark-sql \ >> --packages org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.8.1 \ >> --conf >> spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions >> \ >> --conf >> spark.sql.catalog.polaris=org.apache.iceberg.spark.SparkCatalog \ >> --conf >> spark.sql.catalog.polaris.catalog-impl=org.apache.iceberg.rest.RESTCatalog \ >> --conf spark.sql.catalog.polaris.uri= >> http://127.0.0.1:8181/api/catalog \ >> --conf spark.sql.catalog.polaris.credential=root:<secret> \ >> --conf spark.sql.catalog.polaris.scope=PRINCIPAL_ROLE:ALL \ >> --conf spark.sql.catalog.polaris.warehouse=polaris_demo >> >> spark-sql (default)> create namespace polaris.foo; >> 25/02/27 09:43:24 WARN RESTSessionCatalog: Iceberg REST client is missing >> the OAuth2 server URI configuration and defaults to >> http://127.0.0.1:8181/api/catalog/v1/oauth/tokens. This automatic >> fallback will be removed in a future Iceberg release.It is recommended to >> configure the OAuth2 endpoint using the 'oauth2-server-uri' property to be >> prepared. This warning will disappear if the OAuth2 endpoint is explicitly >> configured. See https://github.com/apache/iceberg/issues/10537 >> Time taken: 0.688 seconds >> spark-sql (default)> >> >> >> On Thu, Feb 27, 2025 at 12:22 AM Aihua Xu <aihu...@gmail.com> wrote: >> >>> I tested 1.8.1 RC with Snowflake build. I'm seeing the following (I'm >>> not seeing that in 1.7.x). >>> >>> "exception": "java.io.IOException: *Authorization header is missing*\n\tat >>> org.apache.polaris.service.dropwizard.auth.PolarisPrincipalAuthenticator.filter(PolarisPrincipalAuthenticator.java:43)\n\tat >>> org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:108)\n\tat >>> org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:44)\n\tat >>> org.glassfish.jersey.process.internal.Stages.process(Stages.java:173)\n\tat >>> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:248). >>> >>> Does anyone know which change could introduce this? >>> >>> On Wed, Feb 26, 2025 at 5:16 AM Péter Váry <peter.vary.apa...@gmail.com> >>> wrote: >>> >>>> +1 >>>> >>>> checked the signatures, checksums >>>> build and run some tests >>>> >>>> Amogh Jahagirdar <2am...@gmail.com> ezt írta (időpont: 2025. febr. >>>> 26., Sze, 6:11): >>>> >>>>> +1 (binding) >>>>> >>>>> Verified signatures, checksum, RAT checks. >>>>> Ran build and test with JDK17 >>>>> >>>>> Thanks, >>>>> Amogh Jahagirdar >>>>> >>>>> On Wed, Feb 26, 2025 at 2:30 AM Honah J. <hon...@apache.org> wrote: >>>>> >>>>>> +1 (binding) >>>>>> >>>>>> - Checked signatures and checksum >>>>>> - Checked license >>>>>> - Full Build and Test >>>>>> >>>>>> Best regards, >>>>>> Honah >>>>>> >>>>>> On Tue, Feb 25, 2025 at 10:52 AM Russell Spitzer < >>>>>> russell.spit...@gmail.com> wrote: >>>>>> >>>>>>> +1 >>>>>>> Checked Sigs and Checksum >>>>>>> Ran Rat >>>>>>> Ran full build/test >>>>>>> >>>>>>> >>>>>>> On Tue, Feb 25, 2025 at 11:30 AM Driesprong, Fokko >>>>>>> <fo...@driesprong.frl> wrote: >>>>>>> >>>>>>>> +1 (binding) >>>>>>>> >>>>>>>> - Checked signatures and checksum >>>>>>>> - Checked licenses >>>>>>>> - Spotchecked NOTICE/LICENSE >>>>>>>> >>>>>>>> Kind regards, >>>>>>>> Fokko >>>>>>>> >>>>>>>> Op di 25 feb 2025 om 16:56 schreef Kevin Liu <kevinjq...@apache.org >>>>>>>> >: >>>>>>>> >>>>>>>>> +1 (non-binding) >>>>>>>>> >>>>>>>>> I followed "How to Verify a Release" >>>>>>>>> <https://iceberg.apache.org/how-to-release/#how-to-verify-a-release> >>>>>>>>> . >>>>>>>>> Checked out artifact from SVN, >>>>>>>>> ``` >>>>>>>>> svn checkout >>>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.8.1-rc1/ >>>>>>>>> . >>>>>>>>> ``` >>>>>>>>> >>>>>>>>> Verified >>>>>>>>> * Signature Good >>>>>>>>> * Checksum Ok >>>>>>>>> * RAT check passed. 1 unrelated error message >>>>>>>>> ``` >>>>>>>>> ERROR: Ignored 0 lines in your exclusion files as comments or >>>>>>>>> empty lines. >>>>>>>>> ``` >>>>>>>>> * Build + test passed, running on Java 17.0.6 (openjdk 17.0.6 >>>>>>>>> 2023-01-17 LTS) on M1 >>>>>>>>> * Ran a few examples on Spark >>>>>>>>> * Ran pyiceberg integration tests, >>>>>>>>> https://github.com/kevinjqliu/iceberg-python/pull/11 >>>>>>>>> >>>>>>>>> I ran the tests both with and without the docker daemon. Without >>>>>>>>> docker, a few tests failed in `iceberg-aws`, `iceberg-azure`, and >>>>>>>>> `iceberg-kafka-connect`. There's already an issue to track this at >>>>>>>>> https://github.com/apache/iceberg/issues/12236. >>>>>>>>> I'm also continuing to see the flakey test for `iceberg-core`'s >>>>>>>>> `testConcurrentFastAppends` test. I believe this is a local issue >>>>>>>>> with my >>>>>>>>> machine. >>>>>>>>> >>>>>>>>> Thanks for running the release, Eduard! >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Kevin Liu >>>>>>>>> >>>>>>>>> On Tue, Feb 25, 2025 at 4:23 AM Jean-Baptiste Onofré < >>>>>>>>> j...@nanthrax.net> wrote: >>>>>>>>> >>>>>>>>>> +1 (non binding) >>>>>>>>>> >>>>>>>>>> - Hash and checksum are good >>>>>>>>>> - LICENSE and NOTICE are OK in different distributed artifacts >>>>>>>>>> (source >>>>>>>>>> distribution, aws bundle, etc) >>>>>>>>>> - ASF header present in all expected files >>>>>>>>>> - No binary file found in the source distribution >>>>>>>>>> - Did quick smoke tests >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Regards >>>>>>>>>> JB >>>>>>>>>> >>>>>>>>>> On Mon, Feb 24, 2025 at 1:46 PM Eduard Tudenhoefner >>>>>>>>>> <etudenhoef...@gmail.com> wrote: >>>>>>>>>> > >>>>>>>>>> > Hi Everyone, >>>>>>>>>> > >>>>>>>>>> > I propose that we release the following RC as the official >>>>>>>>>> Apache Iceberg 1.8.1 release. >>>>>>>>>> > >>>>>>>>>> > The commit ID is 9ce0fcf0af7becf25ad9fc996c3bad2afdcfd33d >>>>>>>>>> > * This corresponds to the tag: apache-iceberg-1.8.1-rc1 >>>>>>>>>> > * >>>>>>>>>> https://github.com/apache/iceberg/commits/apache-iceberg-1.8.1-rc1 >>>>>>>>>> > * >>>>>>>>>> https://github.com/apache/iceberg/tree/9ce0fcf0af7becf25ad9fc996c3bad2afdcfd33d >>>>>>>>>> > >>>>>>>>>> > The release tarball, signature, and checksums are here: >>>>>>>>>> > * >>>>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.8.1-rc1 >>>>>>>>>> > >>>>>>>>>> > You can find the KEYS file here: >>>>>>>>>> > * https://downloads.apache.org/iceberg/KEYS >>>>>>>>>> > >>>>>>>>>> > Convenience binary artifacts are staged on Nexus. The Maven >>>>>>>>>> repository URL is: >>>>>>>>>> > * >>>>>>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1184/ >>>>>>>>>> > >>>>>>>>>> > Please download, verify, and test. >>>>>>>>>> > >>>>>>>>>> > Please vote in the next 72 hours. >>>>>>>>>> > >>>>>>>>>> > [ ] +1 Release this as Apache Iceberg 1.8.1 >>>>>>>>>> > [ ] +0 >>>>>>>>>> > [ ] -1 Do not release this because... >>>>>>>>>> > >>>>>>>>>> > Only PMC members have binding votes, but other community >>>>>>>>>> members are encouraged to cast >>>>>>>>>> > non-binding votes. This vote will pass if there are 3 binding >>>>>>>>>> +1 votes and more binding >>>>>>>>>> > +1 votes than -1 votes. >>>>>>>>>> > >>>>>>>>>> >>>>>>>>>
