Hi ,
I am trying to setup the hive metastore and mysql DB connection.
I have a spark cluster and I ran some programs and I have data stored in
some hive tables.
Now I want to store this data into Mysql so that it is available for
further processing.
I setup the hive-site.xml file.
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>hive.semantic.analyzer.factory.impl</name>
<value>org.apache.hcatalog.cli.HCatSemanticAnalyzerFactory</value>
</property>
<property>
<name>hive.metastore.sasl.enabled</name>
<value>false</value>
</property>
<property>
<name>hive.server2.authentication</name>
<value>NONE</value>
</property>
<property>
<name>hive.server2.enable.doAs</name>
<value>true</value>
</property>
<property>
<name>hive.warehouse.subdir.inherit.perms</name>
<value>true</value>
</property>
<property>
<name>hive.metastore.schema.verification</name>
<value>false</value>
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://<*ip address*
>:3306/metastore_db?createDatabaseIfNotExist=true</value>
<description>metadata is stored in a MySQL server</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>MySQL JDBC driver class</description>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value></value>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/${user.name}/hive-warehouse</value>
<description>location of default database for
the warehouse</description>
</property>
</configuration>
--------------
My mysql server is on a separate server than where my spark server is . If
I use mySQLWorkbench , I use a SSH connection with a certificate file to
connect .
How do I specify all that information from spark to the DB ?
I want to store the data generated by my spark program into mysql.
Thanks
_R
