Hi all, I have a spark application that was working in 1.5.2, but now has a
problem in 1.6.2.
Here is an example:
val conf = new SparkConf()
.setMaster("spark://10.0.2.15:7077")
.setMaster("local")
.set("spark.cassandra.connection.host", "10.0.2.15")
.setAppName("spark-sql-dataexample");
val hiveSqlContext = new HiveContext(SparkContext.getOrCreate(conf));
//Registering tables....
var query = """OBJ_TAB""".stripMargin;
val options = Map(
"driver" -> "org.postgresql.Driver",
"url" -> "jdbc:postgresql://127.0.0.1:5432/DB",
"user" -> "postgres",
"password" -> "postgres",
"dbtable" -> query);
import hiveSqlContext.implicits._;
val df: DataFrame =
hiveSqlContext.read.format("jdbc").options(options).load();
df.registerTempTable("V_OBJECTS");
val optionsC = Map("table"->"data_tab", "keyspace"->"data");
val stats : DataFrame =
hiveSqlContext.read.format("org.apache.spark.sql.cassandra").options(optionsC).load();
//stats.foreach { x => println(x) }
stats.registerTempTable("V_DATA");
//START HIVE SERVER
HiveThriftServer2.startWithContext(hiveSqlContext);
Now, from app I can perform queries and joins over the 2 registered table,
but if I connect to port 10000 via beeline, I see no registered tables.
show tables is empty.
I'm using embedded DERBY DB, but this was working in 1.5.2.
Any suggestion?
Thanks!!!!