Hi Badrinath,
This is a very valid question.
The option of getting a ticket before being authorised is clearly not going
to work here as any authentication of that nature applies to the
environment where both Hive and Spark co-exist. So the question has to move
to how we can authenticate connection to remote Hive with beeline.
If you look at the Spark connection syntax we have
def loadTableFromHiveJDBC(dataFrame,tableName)
try:
house_df = spark.read. \
format("jdbc"). \
option("url", config['hiveVariables']['hive_url']). \
option("dbtable", tableName). \
option("user", config['hiveVariables']['hive_user']). \
option("password", config['hiveVariables']['hive_password']). \
option("driver", config['hiveVariables']['hive_driver']). \
option("fetchsize", config['hiveVariables']['fetchsize']). \
load()
return house_df
except Exception as e:
print(f"""{e}, quitting""")
sys.exit(1)
So we no longer require the option("user",..) and option ("password", ...)
In simplest form the above is equivalent to below thrift connection
beeline -u jdbc:hive2://HOST:PORT/default org.apache.hive.jdbc.HiveDriver
-n hduser -p hduser
In other words
beeline -u <URL> <driver> -n <username> -p <password>
The link you provided states:
beeline -u "jdbc:hive2://HOST:PORT/default;principal=hive/[email protected]"
So we need to carry the authentication through hive_url and that needs to
be valid on the 'remote environment' that Hive is running!
Do you have a kerberized Hive that you can test this please, assuming you
have all the details for the principal?
Thanks
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Thu, 28 Jan 2021 at 13:30, badrinath patchikolla <
[email protected]> wrote:
> Hi Mich,
>
> Is there any possible way to connect Hive JDBC through Kerberos
> Authentication Type in Spark JDBC?
>
>
> https://docs.cloudera.com/runtime/7.2.2/securing-hive/topics/hive_remote_data_access.html
> .
>
>
> Thanks,
> Badrinath
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: [email protected]
>
>
