I'm running within our integration tests. I'm able to trace and find the breaking change: https://github.com/apache/iceberg/pull/11992.
Seems we need to make the following changes to pass the token: https://github.com/apache/iceberg/pull/12415/files. Alex, can you help take a look at this? if (hasCredential) { authResponse = OAuth2Util.fetchToken( initClient, initHeaders, credential, scope, oauth2ServerUri, optionalOAuthParams); Map<String, String> authHeaders = RESTUtil.merge(initHeaders, OAuth2Util.authHeaders(authResponse.token())); config = fetchConfig(initClient, authHeaders, props); } else { authResponse = null; Map<String, String> authHeaders = RESTUtil.merge(initHeaders, OAuth2Util.authHeaders(initToken)); config = fetchConfig(initClient, authHeaders, props); } On Thu, Feb 27, 2025 at 12:46 AM Alex Dutra <alex.du...@dremio.com.invalid> wrote: > Hi Aihua, > > I just tested 1.8.1 with Polaris OSS and I am not seeing anything > different. Can you share your setup? > > Below is my Spark setup. > > Thanks, > > Alex > > ./gradlew run > > token=$(curl -s http://localhost:8181/api/catalog/v1/oauth/tokens \ > --user root:<secret> \ > -d grant_type=client_credentials \ > -d scope=PRINCIPAL_ROLE:ALL | sed -n > 's/.*"access_token":"\([^"]*\)".*/\1/p') > > curl -s -H "Authorization: Bearer ${token}" \ > -H 'Accept: application/json' \ > -H 'Content-Type: application/json' \ > http://localhost:8181/api/management/v1/catalogs \ > -d '{ > "catalog": { > "name": "polaris_demo", > "type": "INTERNAL", > "readOnly": false, > "properties": { > "default-base-location": "file:///tmp/polaris/" > }, > "storageConfigInfo": { > "storageType": "FILE", > "allowedLocations": [ > "file:///tmp" > ] > } > } > }' > > > spark-sql \ > --packages org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.8.1 \ > --conf > spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions > \ > --conf spark.sql.catalog.polaris=org.apache.iceberg.spark.SparkCatalog > \ > --conf > spark.sql.catalog.polaris.catalog-impl=org.apache.iceberg.rest.RESTCatalog \ > --conf spark.sql.catalog.polaris.uri=http://127.0.0.1:8181/api/catalog > \ > --conf spark.sql.catalog.polaris.credential=root:<secret> \ > --conf spark.sql.catalog.polaris.scope=PRINCIPAL_ROLE:ALL \ > --conf spark.sql.catalog.polaris.warehouse=polaris_demo > > spark-sql (default)> create namespace polaris.foo; > 25/02/27 09:43:24 WARN RESTSessionCatalog: Iceberg REST client is missing > the OAuth2 server URI configuration and defaults to > http://127.0.0.1:8181/api/catalog/v1/oauth/tokens. This automatic > fallback will be removed in a future Iceberg release.It is recommended to > configure the OAuth2 endpoint using the 'oauth2-server-uri' property to be > prepared. This warning will disappear if the OAuth2 endpoint is explicitly > configured. See https://github.com/apache/iceberg/issues/10537 > Time taken: 0.688 seconds > spark-sql (default)> > > > On Thu, Feb 27, 2025 at 12:22 AM Aihua Xu <aihu...@gmail.com> wrote: > >> I tested 1.8.1 RC with Snowflake build. I'm seeing the following (I'm not >> seeing that in 1.7.x). >> >> "exception": "java.io.IOException: *Authorization header is missing*\n\tat >> org.apache.polaris.service.dropwizard.auth.PolarisPrincipalAuthenticator.filter(PolarisPrincipalAuthenticator.java:43)\n\tat >> org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:108)\n\tat >> org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:44)\n\tat >> org.glassfish.jersey.process.internal.Stages.process(Stages.java:173)\n\tat >> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:248). >> >> Does anyone know which change could introduce this? >> >> On Wed, Feb 26, 2025 at 5:16 AM Péter Váry <peter.vary.apa...@gmail.com> >> wrote: >> >>> +1 >>> >>> checked the signatures, checksums >>> build and run some tests >>> >>> Amogh Jahagirdar <2am...@gmail.com> ezt írta (időpont: 2025. febr. 26., >>> Sze, 6:11): >>> >>>> +1 (binding) >>>> >>>> Verified signatures, checksum, RAT checks. >>>> Ran build and test with JDK17 >>>> >>>> Thanks, >>>> Amogh Jahagirdar >>>> >>>> On Wed, Feb 26, 2025 at 2:30 AM Honah J. <hon...@apache.org> wrote: >>>> >>>>> +1 (binding) >>>>> >>>>> - Checked signatures and checksum >>>>> - Checked license >>>>> - Full Build and Test >>>>> >>>>> Best regards, >>>>> Honah >>>>> >>>>> On Tue, Feb 25, 2025 at 10:52 AM Russell Spitzer < >>>>> russell.spit...@gmail.com> wrote: >>>>> >>>>>> +1 >>>>>> Checked Sigs and Checksum >>>>>> Ran Rat >>>>>> Ran full build/test >>>>>> >>>>>> >>>>>> On Tue, Feb 25, 2025 at 11:30 AM Driesprong, Fokko >>>>>> <fo...@driesprong.frl> wrote: >>>>>> >>>>>>> +1 (binding) >>>>>>> >>>>>>> - Checked signatures and checksum >>>>>>> - Checked licenses >>>>>>> - Spotchecked NOTICE/LICENSE >>>>>>> >>>>>>> Kind regards, >>>>>>> Fokko >>>>>>> >>>>>>> Op di 25 feb 2025 om 16:56 schreef Kevin Liu <kevinjq...@apache.org >>>>>>> >: >>>>>>> >>>>>>>> +1 (non-binding) >>>>>>>> >>>>>>>> I followed "How to Verify a Release" >>>>>>>> <https://iceberg.apache.org/how-to-release/#how-to-verify-a-release> >>>>>>>> . >>>>>>>> Checked out artifact from SVN, >>>>>>>> ``` >>>>>>>> svn checkout >>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.8.1-rc1/ >>>>>>>> . >>>>>>>> ``` >>>>>>>> >>>>>>>> Verified >>>>>>>> * Signature Good >>>>>>>> * Checksum Ok >>>>>>>> * RAT check passed. 1 unrelated error message >>>>>>>> ``` >>>>>>>> ERROR: Ignored 0 lines in your exclusion files as comments or empty >>>>>>>> lines. >>>>>>>> ``` >>>>>>>> * Build + test passed, running on Java 17.0.6 (openjdk 17.0.6 >>>>>>>> 2023-01-17 LTS) on M1 >>>>>>>> * Ran a few examples on Spark >>>>>>>> * Ran pyiceberg integration tests, >>>>>>>> https://github.com/kevinjqliu/iceberg-python/pull/11 >>>>>>>> >>>>>>>> I ran the tests both with and without the docker daemon. Without >>>>>>>> docker, a few tests failed in `iceberg-aws`, `iceberg-azure`, and >>>>>>>> `iceberg-kafka-connect`. There's already an issue to track this at >>>>>>>> https://github.com/apache/iceberg/issues/12236. >>>>>>>> I'm also continuing to see the flakey test for `iceberg-core`'s >>>>>>>> `testConcurrentFastAppends` test. I believe this is a local issue with >>>>>>>> my >>>>>>>> machine. >>>>>>>> >>>>>>>> Thanks for running the release, Eduard! >>>>>>>> >>>>>>>> Best, >>>>>>>> Kevin Liu >>>>>>>> >>>>>>>> On Tue, Feb 25, 2025 at 4:23 AM Jean-Baptiste Onofré < >>>>>>>> j...@nanthrax.net> wrote: >>>>>>>> >>>>>>>>> +1 (non binding) >>>>>>>>> >>>>>>>>> - Hash and checksum are good >>>>>>>>> - LICENSE and NOTICE are OK in different distributed artifacts >>>>>>>>> (source >>>>>>>>> distribution, aws bundle, etc) >>>>>>>>> - ASF header present in all expected files >>>>>>>>> - No binary file found in the source distribution >>>>>>>>> - Did quick smoke tests >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Regards >>>>>>>>> JB >>>>>>>>> >>>>>>>>> On Mon, Feb 24, 2025 at 1:46 PM Eduard Tudenhoefner >>>>>>>>> <etudenhoef...@gmail.com> wrote: >>>>>>>>> > >>>>>>>>> > Hi Everyone, >>>>>>>>> > >>>>>>>>> > I propose that we release the following RC as the official >>>>>>>>> Apache Iceberg 1.8.1 release. >>>>>>>>> > >>>>>>>>> > The commit ID is 9ce0fcf0af7becf25ad9fc996c3bad2afdcfd33d >>>>>>>>> > * This corresponds to the tag: apache-iceberg-1.8.1-rc1 >>>>>>>>> > * >>>>>>>>> https://github.com/apache/iceberg/commits/apache-iceberg-1.8.1-rc1 >>>>>>>>> > * >>>>>>>>> https://github.com/apache/iceberg/tree/9ce0fcf0af7becf25ad9fc996c3bad2afdcfd33d >>>>>>>>> > >>>>>>>>> > The release tarball, signature, and checksums are here: >>>>>>>>> > * >>>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.8.1-rc1 >>>>>>>>> > >>>>>>>>> > You can find the KEYS file here: >>>>>>>>> > * https://downloads.apache.org/iceberg/KEYS >>>>>>>>> > >>>>>>>>> > Convenience binary artifacts are staged on Nexus. The Maven >>>>>>>>> repository URL is: >>>>>>>>> > * >>>>>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1184/ >>>>>>>>> > >>>>>>>>> > Please download, verify, and test. >>>>>>>>> > >>>>>>>>> > Please vote in the next 72 hours. >>>>>>>>> > >>>>>>>>> > [ ] +1 Release this as Apache Iceberg 1.8.1 >>>>>>>>> > [ ] +0 >>>>>>>>> > [ ] -1 Do not release this because... >>>>>>>>> > >>>>>>>>> > Only PMC members have binding votes, but other community members >>>>>>>>> are encouraged to cast >>>>>>>>> > non-binding votes. This vote will pass if there are 3 binding +1 >>>>>>>>> votes and more binding >>>>>>>>> > +1 votes than -1 votes. >>>>>>>>> > >>>>>>>>> >>>>>>>>
