Hi: I want to use spark jdbc to read kerberized impala tables, like: ``` val impalaUrl = "jdbc:impala://<host_imapal_deamon>:21050;AuthMech=1;KrbRealm=REALM.COM;KrbHostFQDN=<host_impala_deamon>;KrbServiceName=impala" spark.read.jdbc(impalaUrl) ```
As we know, spark will read impala data by executor rather than driver, so throw excepting: javax.security.sasl.SaslException: GSS initiate failed ``` Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt) at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147) at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122) at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187) at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ... 20 common frames omitted ``` Ony way to solve this problem is set jaas.conf by "java.security.auth.login.config" property, This is jaas.conf: ``` Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true doNotPrompt=true useTicketCache=true principal="test" keyTab="/home/keytab/user.keytab"; }; ``` Then set spark.executor.extraJavaOptions like : ``` --conf "spark.executor.extraJavaOptions=-Djava.security.auth.login.config=/data/disk1/spark-jdbc-impala/conf/jaas.conf -Djavax.security.auth.useSubjectCredsOnly=false" ``` This way required absolute jaas.conf file and keyTab file, in other words, these files must be placed in the same path and on each node, Is there a better way? Please help. Regards eab...@163.com