At the moment I can't think of any better but we've added
custom JdbcConnectionProvider API in Spark 3.1.
Hope that will make life easier in the future...
G
On Tue, Dec 8, 2020 at 3:55 AM eab...@163.com <eab...@163.com> wrote:
> Hi:
>
> I want to use spark jdbc to read kerberized impala tables, like:
> ```
> val impalaUrl =
> "jdbc:impala://<host_imapal_deamon>:21050;AuthMech=1;KrbRealm=REALM.COM
> ;KrbHostFQDN=<host_impala_deamon>;KrbServiceName=impala"
> spark.read.jdbc(impalaUrl)
> ```
>
> As we know, spark will read impala data by executor rather than driver, so
> throw excepting: javax.security.sasl.SaslException: GSS initiate failed
>
> ```
> Caused by: org.ietf.jgss.GSSException: No valid credentials provided
> (Mechanism level: Failed to find any Kerberos tgt)
> at
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
> at
> sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192)
> ... 20 common frames omitted
>
> ```
>
> Ony way to solve this problem is set jaas.conf by
> "java.security.auth.login.config" property,
>
> This is jaas.conf:
>
> ```
> Client {
> com.sun.security.auth.module.Krb5LoginModule required
> useKeyTab=true
> doNotPrompt=true
> useTicketCache=true
> principal="test"
> keyTab="/home/keytab/user.keytab";
> };
>
> ```
>
> Then set spark.executor.extraJavaOptions like :
> ```
> --conf
> "spark.executor.extraJavaOptions=-Djava.security.auth.login.config=/data/disk1/spark-jdbc-impala/conf/jaas.conf
> -Djavax.security.auth.useSubjectCredsOnly=false"
> ```
>
> This way required absolute jaas.conf file and keyTab file, in other words,
> these files must be placed in the same path and on each node, Is there a
> better way?
>
> Please help.
>
> Regards
>
>
> ------------------------------
> eab...@163.com
>
