[
https://issues.apache.org/jira/browse/HIVE-25575?focusedWorklogId=724822&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-724822
]
ASF GitHub Bot logged work on HIVE-25575:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Feb/22 22:25
Start Date: 10/Feb/22 22:25
Worklog Time Spent: 10m
Work Description: nrg4878 commented on a change in pull request #3006:
URL: https://github.com/apache/hive/pull/3006#discussion_r802935087
##########
File path: common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
##########
@@ -4046,6 +4046,12 @@ private static void
populateLlapDaemonVarsSet(Set<String> llapDaemonVarsSetLocal
HIVE_SERVER2_THRIFT_HTTP_COMPRESSION_ENABLED("hive.server2.thrift.http.compression.enabled",
true,
"Enable thrift http compression via Jetty compression support"),
+ // JWT Auth configs
+
HIVE_SERVER2_THRIFT_HTTP_JWT_JWKS_URL("hive.server2.thrift.http.jwt.jwks.url",
"",
Review comment:
thrift.http part of the property would be a bit confusing, as we plan on
adding such support for binary mode as well. So the JWKS URL would be the same
in both cases.
Similar to what we used for other auth-specific properties, maybe use
something like "hive.server2.authentication.jwt.jwks.url"?
##########
File path:
jdbc/src/java/org/apache/hive/jdbc/auth/jwt/HttpJwtAuthRequestInterceptor.java
##########
@@ -0,0 +1,49 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hive.jdbc.auth.jwt;
+
+import org.apache.hive.jdbc.HttpRequestInterceptorBase;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpRequest;
+import org.apache.http.client.CookieStore;
+import org.apache.http.protocol.HttpContext;
+
+import java.util.Map;
+
+/**
+ * This implements the logic to intercept the HTTP requests from the Hive Jdbc
connection
+ * and adds JWT auth header.
+ */
+public class HttpJwtAuthRequestInterceptor extends HttpRequestInterceptorBase {
+
+ private static final String BEARER = "Bearer ";
Review comment:
isnt there a constant defined for this already? if not, is there a
better location such constant because it is used in browser auth as well ?
Thanks
##########
File path:
itests/hive-unit/src/test/java/org/apache/hive/service/auth/jwt/TestHttpJwtAuthentication.java
##########
@@ -0,0 +1,211 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hive.service.auth.jwt;
+
+import com.nimbusds.jose.JWSAlgorithm;
+import com.nimbusds.jose.JWSHeader;
+import com.nimbusds.jose.JWSSigner;
+import com.nimbusds.jose.crypto.RSASSASigner;
+import com.nimbusds.jose.jwk.RSAKey;
+import com.nimbusds.jwt.JWTClaimsSet;
+import com.nimbusds.jwt.SignedJWT;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.conf.HiveConf.ConfVars;
+import org.apache.hive.jdbc.HiveConnection;
+import org.apache.hive.jdbc.Utils;
+import org.apache.hive.jdbc.miniHS2.MiniHS2;
+import org.junit.AfterClass;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import java.io.File;
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Path;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+
+
+public class TestHttpJwtAuthentication {
+ private static final Map<String, String> DEFAULTS = new
HashMap<>(System.getenv());
+ private static Map<String, String> envMap;
+
+ private static final File jwtAuthorizedKeyFile =
+ new File("src/test/resources/auth.jwt/jwt-authorized-key.json");
+ private static final File jwtUnauthorizedKeyFile =
+ new File("src/test/resources/auth.jwt/jwt-unauthorized-key.json");
+ private static final File jwtVerificationJWKSFile =
+ new File("src/test/resources/auth.jwt/jwt-verification-jwks.json");
+
+ public static final String USER_1 = "USER_1";
+
+ private static MiniHS2 miniHS2;
+
+ @BeforeClass
+ public static void makeEnvModifiable() throws Exception {
+ envMap = new HashMap<>();
+ Class<?> envClass = Class.forName("java.lang.ProcessEnvironment");
+ Field theEnvironmentField = envClass.getDeclaredField("theEnvironment");
+ Field theUnmodifiableEnvironmentField =
envClass.getDeclaredField("theUnmodifiableEnvironment");
+ removeStaticFinalAndSetValue(theEnvironmentField, envMap);
+ removeStaticFinalAndSetValue(theUnmodifiableEnvironmentField, envMap);
+ }
+
+ private static void removeStaticFinalAndSetValue(Field field, Object value)
throws Exception {
+ field.setAccessible(true);
+ Field modifiersField = Field.class.getDeclaredField("modifiers");
+ modifiersField.setAccessible(true);
+ modifiersField.setInt(field, field.getModifiers() & ~Modifier.FINAL);
+ field.set(null, value);
+ }
+
+ @Before
+ public void initEnvMap() {
+ envMap.clear();
+ envMap.putAll(DEFAULTS);
+ }
+
+ @BeforeClass
+ public static void setupHS2() throws Exception {
+ HiveConf conf = new HiveConf();
+ conf.setBoolVar(ConfVars.HIVE_SUPPORT_CONCURRENCY, false);
+ conf.setBoolVar(ConfVars.HIVE_SERVER2_LOGGING_OPERATION_ENABLED, false);
+ conf.setBoolVar(ConfVars.HIVESTATSCOLAUTOGATHER, false);
+ conf.setVar(ConfVars.HIVE_SERVER2_AUTHENTICATION, "JWT");
+ // the content of the URL below is the same as jwtVerificationJWKSFile
+ conf.setVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_JWT_JWKS_URL,
+
"https://gist.githubusercontent.com/hsnusonic/d06f2f18a73d1dbbba081e0267467da6/raw/38c2930d134c78320219b838bac4ceee680817bd/jwks.json";);
+ miniHS2 = new MiniHS2.Builder().withConf(conf).withHTTPTransport().build();
+
+ miniHS2.start(new HashMap<>());
+ }
+
+ @AfterClass
+ public static void stopServices() throws Exception {
+ if (miniHS2 != null && miniHS2.isStarted()) {
+ miniHS2.stop();
+ miniHS2.cleanup();
+ miniHS2 = null;
+ MiniHS2.cleanupLocalDir();
+ }
+ }
+
+
+
+ @Test
+ public void testAuthorizedUser() throws Exception {
+ String jwt = generateJWT(USER_1, jwtAuthorizedKeyFile.toPath(),
TimeUnit.MINUTES.toMillis(5));
+ HiveConnection connection = getConnection(jwt, true);
+ assertLoggedInUser(connection, USER_1);
+ connection.close();
+
+ connection = getConnection(jwt, false);
+ assertLoggedInUser(connection, USER_1);
+ connection.close();
+ }
+
+ @Test(expected = SQLException.class)
+ public void testExpiredJwt() throws Exception {
+ String jwt = generateJWT(USER_1, jwtAuthorizedKeyFile.toPath(), 1);
+ Thread.sleep(1);
+ HiveConnection connection = getConnection(jwt, true);
+ }
+
+ @Test(expected = SQLException.class)
+ public void testUnauthorizedUser() throws Exception {
+ String unauthorizedJwt = generateJWT(USER_1,
jwtUnauthorizedKeyFile.toPath(), TimeUnit.MINUTES.toMillis(5));
+ HiveConnection connection = getConnection(unauthorizedJwt, true);
+ }
+
+ @Test(expected = SQLException.class)
+ public void testWithoutJwtProvided() throws Exception {
+ HiveConnection connection = getConnection(null, true);
+ }
+
+ private HiveConnection getConnection(String jwt, Boolean putJwtInEnv) throws
Exception {
+ String url = getJwtJdbcConnectionUrl();
+ if (jwt != null && putJwtInEnv) {
+ System.getenv().put(Utils.JdbcConnectionParams.AUTH_JWT_ENV, jwt);
+ } else if (jwt != null) {
+ url += "jwt=" + jwt;
+ }
+ Class.forName("org.apache.hive.jdbc.HiveDriver");
+ Connection connection = DriverManager.getConnection(url, null, null);
+ return (HiveConnection) connection;
+ }
+
+
Review comment:
nit: extra empty line
##########
File path: pom.xml
##########
@@ -354,6 +355,11 @@
<artifactId>parquet-hadoop-bundle</artifactId>
<version>${parquet.version}</version>
</dependency>
+ <dependency>
+ <groupId>com.nimbusds</groupId>
+ <artifactId>nimbus-jose-jwt</artifactId>
+ <version>${nimbus-jose-jwt.version}</version>
+ </dependency>
Review comment:
can we ensure that it does not pull in other libraries that we do not
need?
##########
File path:
service/src/java/org/apache/hive/service/auth/jwt/URLBasedJWKSProvider.java
##########
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hive.service.auth.jwt;
+
+import com.nimbusds.jose.JWSHeader;
+import com.nimbusds.jose.jwk.JWK;
+import com.nimbusds.jose.jwk.JWKMatcher;
+import com.nimbusds.jose.jwk.JWKSelector;
+import com.nimbusds.jose.jwk.JWKSet;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URL;
+import java.text.ParseException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link JWKSProvider} which reads JWKS from URL.
+ */
+public class URLBasedJWKSProvider implements JWKSProvider {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(URLBasedJWKSProvider.class.getName());
+ private final HiveConf conf;
+ private List<JWKSet> jwkSets = new ArrayList<>();
+
+ public URLBasedJWKSProvider(HiveConf conf) {
+ this.conf = conf;
+ loadJWKSets();
+ }
+
+ private void loadJWKSets() {
+ String jwksURL = HiveConf.getVar(conf,
HiveConf.ConfVars.HIVE_SERVER2_THRIFT_HTTP_JWT_JWKS_URL);
+ List<String> jwksURLs =
Arrays.stream(jwksURL.split(",")).collect(Collectors.toList());
+ for (String urlString : jwksURLs) {
+ try {
+ URL url = new URL(urlString);
+ jwkSets.add(JWKSet.load(url));
+ LOG.info("Loaded JWKS from " + urlString);
+ } catch (IOException | ParseException e) {
+ LOG.info("Failed to retrieve JWKS from {}: {}", urlString,
e.getMessage());
+ }
+ }
+ }
+
+ @Override
+ public List<JWK> getJWKs(JWSHeader header) {
+ List<JWK> jwks = new ArrayList<>();
+ JWKSelector selector = new JWKSelector(JWKMatcher.forJWSHeader(header));
Review comment:
This is for my understanding. Does this retrieve the key based on the
kid?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 724822)
Time Spent: 0.5h (was: 20m)
> Add support for JWT authentication in HTTP mode
> -----------------------------------------------
>
> Key: HIVE-25575
> URL: https://issues.apache.org/jira/browse/HIVE-25575
> Project: Hive
> Issue Type: New Feature
> Components: HiveServer2, JDBC
> Affects Versions: 4.0.0
> Reporter: Shubham Chaurasia
> Assignee: Yu-Wen Lai
> Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> It would be good to support JWT auth mechanism in hive. In order to implement
> it, we would need the following -
> On HS2 side -
> 1. Accept JWT in Authorization: Bearer header.
> 2. Fetch JWKS from a public endpoint to verify JWT signature, to start with
> we can fetch on HS2 start up.
> 3. Verify JWT Signature.
> On JDBC Client side -
> 1. Hive jdbc client should be able to accept jwt in JDBC url. (will add more
> details)
> 2. Client should also be able to pick up JWT from an env var if it's defined.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
