>>> val hc = new org.apache.spark.sql.hive.HiveContext(sc)
>>> val result = hc.sql("select * from hivedb.hivetable")
>>> result.registerTempTable("myTest")
You should not create hiveContext directly as above, just use sqlContext
created by zeppelin.
LINZ, Arnaud <al...@bouyguestelecom.fr>于2018年2月5日周一 下午9:15写道：
> Thanks for your answer, but it does not address my problem.
>
> - I don’t create sqlContext, I use the one provided by Zeppelin. But
> sqlContext is not a hive Context and cannot access hive metastore.
> - Zeppelin can see my hive conf files, and selecting tables through a
> created hiveContext works. But I cannot visualize them in the %sql
> graphical interpretor.
>
>
>
> *De :* Jeff Zhang [mailto:zjf...@gmail.com]
> *Envoyé :* lundi 5 février 2018 14:01
> *À :* users@zeppelin.apache.org
> *Objet :* Re: How to have a native graphical representation (%sql) of a
> HiveContext?
>
>
>
>
>
> 1. Don't create sqlContext in zeppelin as zeppelin will create that for
> you, and %sql use the sqlContext created by zeppelin itself.
>
> 2. Make sure you have hive-site.xml under SPARK_CONF_DIR if you want to
> use hiveContext. Otherwise spark will use single user derby instance which
> is not for production, and will cause conflicts when you create multiple
> spark interpreter in one zeppelin instance.
>
>
>
>
>
> LINZ, Arnaud <al...@bouyguestelecom.fr>于2018年2月5日周一 下午8:33写道：
>
> Hello,
>
>
>
> I’m trying to install Zeppelin (0.7.2) on my CDH cluster, and I am unable
> to connect the sql + graphical representations of the %sql interpreter
> with my Hive data, and more surprisingly I really can’t find any good
> source on the internet (apache zeppelin documentation or stack overflow)
> that gives a practical answer about how to do this.
>
> Most of the time, the data comes from compressed Hive tables and not plain
> hdfs text files ; so using a hive context is far more convenient than a
> plain spark sql context.
>
>
>
> The following :
>
> %spark
>
> val hc = new org.apache.spark.sql.hive.HiveContext(sc)
>
> val result = hc.sql("select * from hivedb.hivetable")
>
> result.registerTempTable("myTest")
>
>
>
> works but no myTest table is available in the following %sql interpreter :
>
> %sql
>
> select * from myTest
>
> org.apache.spark.sql.AnalysisException: Table not found: myTest;
>
>
>
>
>
> However the following :
>
> %pyspark
>
> result = sqlContext.read.text("hdfs://cluster/test.txt")
>
> result.registerTempTable("mySqlTest")
>
>
>
> works as the %sql interpreter is “plugged” to the sqlContext
>
>
>
> but
>
> result = sqlContext.sql("select * from hivedb.hivetable") does not work
> as the sqlContext is not a hive context.
>
>
>
> I have set zeppelin.spark.useHiveContext to true, but it seems to have no
> effect (btw, it was more of a wild guess since the documentation is not
> giving much detail on parameters and context configuration)
>
>
>
> Can you direct me towards how to configure the context used by the %sql
> interpreter?
>
>
>
> Best regards,
>
> Arnaud
>
>
>
> PS : %spark and %sql interpreter conf:
>
>
>
> master yarn-client
>
> spark.app.name Zeppelin
>
> spark.cores.max
>
> spark.executor.memory 5g
>
> zeppelin.R.cmd R
>
> zeppelin.R.image.width 100%
>
> zeppelin.R.knitr true
>
> zeppelin.R.render.options out.format = 'html', comment = NA, echo =
> FALSE, results = 'asis', message = F, warning = F
>
> zeppelin.dep.additionalRemoteRepository spark-packages,
> http://dl.bintray.com/spark-packages/maven,false;
>
> zeppelin.dep.localrepo local-repo
>
> zeppelin.interpreter.localRepo /opt/zeppelin/local-repo/2CYVF45A9
>
> zeppelin.interpreter.output.limit 102400
>
> zeppelin.pyspark.python /usr/bin/pyspark
>
> zeppelin.spark.concurrentSQL true
>
> zeppelin.spark.importImplicit true
>
> zeppelin.spark.maxResult 1000
>
> zeppelin.spark.printREPLOutput true
>
> zeppelin.spark.sql.stacktrace true
>
> zeppelin.spark.useHiveContext true
>
>
> ------------------------------
>
>
> L'intégrité de ce message n'étant pas assurée sur internet, la société
> expéditrice ne peut être tenue responsable de son contenu ni de ses pièces
> jointes. Toute utilisation ou diffusion non autorisée est interdite. Si
> vous n'êtes pas destinataire de ce message, merci de le détruire et
> d'avertir l'expéditeur.
>
> The integrity of this message cannot be guaranteed on the Internet. The
> company that sent this message cannot therefore be held liable for its
> content nor attachments. Any unauthorized use or dissemination is
> prohibited. If you are not the intended recipient of this message, then
> please delete it and notify the sender.
>
>
