I hope Rui (in cc) will be able to help you. Best,
Dawid
On 12/10/2021 15:32, Andrew Otto wrote:
> Hello,
>
> I'm trying to use HiveCatalog with Kerberos. Our Hadoop cluster, our
> Hive Metastore, and our Hive Server are kerberized. I can
> successfully submit Flink jobs to Yarn authenticated as my users using
> a cached ticket, as well as using a keytab.
>
> However, I can't seem to register a HiveCatalog with my
> TableEnvironment. Here's my code:
> import org.apache.flink.table.api._
> import org.apache.flink.table.catalog.hive.HiveCatalog
>
> val settings = EnvironmentSettings.newInstance().inStreamingMode().build()
> val tableEnv = TableEnvironment.create(settings)
> val catalog = new HiveCatalog("analytics_hive", "flink_test",
> "/etc/hive/conf")
> tableEnv.registerCatalog("analytics_hive", catalog)
>
> Which causes an exception:
> Caused by: java.lang.reflect.InvocationTargetException:
> org.apache.hadoop.hive.metastore.api.MetaException: Could not connect
> to meta store using any of the URIs provided. Most recent failure:
> org.apache.thrift.transport.TTransportException: GSS initiate failed
>
> (Full stacktrace here
> <https://gist.github.com/ottomata/79fbad1b97efebd9c71d1bf11d171ade>.)
>
> The same error happens if I try to submit this job using my cached
> kerberos ticket, or with a keytab.
> I have also tried wrapping the HiveCatalog in a Hadoop
> UserGroupInformation PrivilegedExceptionAction as described here
> <https://blog.csdn.net/weibokong789/article/details/106427481> and got
> the same result (no real idea what I'm doing here, just trying some
> things.)
>
> Is there something more I have to do to use HiveCatalog with a
> kerberized Hive Metastore? Should Flink support this out of the box?
>
> Thanks!
> - Andrew Otto
> SRE, Wikimedia Foundation
>
>
OpenPGP_signature
Description: OpenPGP digital signature
