Using Hivecontext solved it.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/spark-sql-join-sql-fails-after-sqlCtx-cacheTable-tp16893p21807.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---
org.apache.hadoop.hive which is not available.
It may be completely missing from the current classpath, or the version on
the classpath might be incompatible with the version used when compiling
HiveContext.class.
That entry seems to have slain the compiler. Shall I replay
your session? I can re-run
Thank for pointing that out.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/spark-sql-join-sql-fails-after-sqlCtx-cacheTable-tp16893p16933.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---
> Hmm... I thought HiveContext will only worki if Hive is present. I am
> curious
> to know when to use HiveContext and when to use SqlContext.
>
http://spark.apache.org/docs/latest/sql-programming-guide.html#getting-started
TLDR; Always use HiveContext if your application does not have a depende
Hmm... I thought HiveContext will only worki if Hive is present. I am curious
to know when to use HiveContext and when to use SqlContext.
Thanks & Regards
Tridib
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/spark-sql-join-sql-fails-after-sqlCtx-cacheTabl
Hi Tridib,
I changed SQLContext to HiveContext and it started working. These are steps
I used.
val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc)
val person = sqlContext.jsonFile("json/person.json")
person.printSchema()
person.registerTempTable("person")
val address = sqlContext.jsonF
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
val personPath = "/hdd/spark/person.json"
val person = sqlContext.jsonFile(personPath)
person.printSchema()
person.registerTempTable("person")
val addressPath = "/hdd/spark/address.json"
val address = sqlContext.jsonFile(addressPath)
address.
Could you please provide some of your code, and the sample json files you
use ?
Regards,
Olivier.
2014-10-21 5:45 GMT+02:00 tridib :
> Hello Experts,
> I have two tables build using jsonFile(). I can successfully run join query
> on these tables. But once I cacheTable(), all join query fails?
>
Hello Experts,
I have two tables build using jsonFile(). I can successfully run join query
on these tables. But once I cacheTable(), all join query fails?
Here is stackstrace:
java.lang.NullPointerException
at
org.apache.spark.sql.columnar.InMemoryRelation.statistics$lzycompute(InMemoryCol
