Hello,
I'm trying to use HiveCatalog with Kerberos. Our Hadoop cluster, our Hive
Metastore, and our Hive Server are kerberized. I can successfully submit
Flink jobs to Yarn authenticated as my users using a cached ticket, as well
as using a keytab.
However, I can't seem to register a HiveCatalog with my TableEnvironment.
Here's my code:
import org.apache.flink.table.api._
import org.apache.flink.table.catalog.hive.HiveCatalog
val settings = EnvironmentSettings.newInstance().inStreamingMode().build()
val tableEnv = TableEnvironment.create(settings)
val catalog = new HiveCatalog("analytics_hive", "flink_test", "/etc/hive/conf")
tableEnv.registerCatalog("analytics_hive", catalog)
Which causes an exception:
Caused by: java.lang.reflect.InvocationTargetException:
org.apache.hadoop.hive.metastore.api.MetaException: Could not connect to
meta store using any of the URIs provided. Most recent failure:
org.apache.thrift.transport.TTransportException: GSS initiate failed
(Full stacktrace here
<https://gist.github.com/ottomata/79fbad1b97efebd9c71d1bf11d171ade>.)
The same error happens if I try to submit this job using my cached kerberos
ticket, or with a keytab.
I have also tried wrapping the HiveCatalog in a Hadoop UserGroupInformation
PrivilegedExceptionAction as described here
<https://blog.csdn.net/weibokong789/article/details/106427481> and got the
same result (no real idea what I'm doing here, just trying some things.)
Is there something more I have to do to use HiveCatalog with a kerberized
Hive Metastore? Should Flink support this out of the box?
Thanks!
- Andrew Otto
SRE, Wikimedia Foundation
