Hi I have a rdd which I want to register as multiple tables based on key
................
val context = new SparkContext(conf)
val sqlContext = new org.apache.spark.sql.hive.HiveContext(context)
import sqlContext.createSchemaRDD
case class KV(key:String,id:String,value:String)
val logsRDD = context.textFile("logs", 10).map{line=>
val Array(key,id,value) = line split ' '
(key,id,value)
}.registerTempTable("KVS")
I want to store the above information to multiple tables based on key
without bringing the entire data to master
Thanks in advance.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/RDD-to-Multiple-Tables-SparkSQL-tp16807.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
