Hey Andrew,
Could you try something like the following? It worked for me to connect to
a kerberized Hive.
UserGroupInformation ugi =
UserGroupInformation.loginUserFromKeytabAndReturnUGI(principal, keytab);
ugi.doAs((PrivilegedAction<Void>) () -> {
HiveCatalog hiveCatalog = new HiveCatalog(...);
tableEnv.registerCatalog("catalog_name", hiveCatalog);
return null;
});
On Thu, Oct 14, 2021 at 8:18 PM Dawid Wysakowicz <dwysakow...@apache.org>
wrote:
> I hope Rui (in cc) will be able to help you.
>
> Best,
>
> Dawid
> On 12/10/2021 15:32, Andrew Otto wrote:
>
> Hello,
>
> I'm trying to use HiveCatalog with Kerberos. Our Hadoop cluster, our Hive
> Metastore, and our Hive Server are kerberized. I can successfully submit
> Flink jobs to Yarn authenticated as my users using a cached ticket, as well
> as using a keytab.
>
> However, I can't seem to register a HiveCatalog with my TableEnvironment.
> Here's my code:
>
> import org.apache.flink.table.api._import
> org.apache.flink.table.catalog.hive.HiveCatalog
> val settings = EnvironmentSettings.newInstance().inStreamingMode().build()val
> tableEnv = TableEnvironment.create(settings)val catalog = new
> HiveCatalog("analytics_hive", "flink_test", "/etc/hive/conf")
> tableEnv.registerCatalog("analytics_hive", catalog)
>
>
> Which causes an exception:
> Caused by: java.lang.reflect.InvocationTargetException:
> org.apache.hadoop.hive.metastore.api.MetaException: Could not connect to
> meta store using any of the URIs provided. Most recent failure:
> org.apache.thrift.transport.TTransportException: GSS initiate failed
>
> (Full stacktrace here
> <https://gist.github.com/ottomata/79fbad1b97efebd9c71d1bf11d171ade>.)
>
> The same error happens if I try to submit this job using my cached
> kerberos ticket, or with a keytab.
> I have also tried wrapping the HiveCatalog in a Hadoop
> UserGroupInformation PrivilegedExceptionAction as described here
> <https://blog.csdn.net/weibokong789/article/details/106427481> and got
> the same result (no real idea what I'm doing here, just trying some things.)
>
> Is there something more I have to do to use HiveCatalog with a kerberized
> Hive Metastore? Should Flink support this out of the box?
>
> Thanks!
> - Andrew Otto
> SRE, Wikimedia Foundation
>
>
>
--
Best regards!
Rui Li
