It is weird, there should be a interpreter log file with name
'zeppelin-interpreter-spark*.log'
Are you sure you are looking at the correct log folder ? Because from the
log you pasted, it is the log of 2017-12-08
LINZ, Arnaud <al...@bouyguestelecom.fr>于2018年2月6日周二 下午6:45写道：
> I’ve been searching for such a log, but I don’t see anything related to
> spark in …/zeppelin/logs
>
> The only logs I have are
>
> zeppelin-$USER-$HOST.log
>
> and
>
> zeppelin-$USER-$HOST.out
>
> but they really don’t contain anything useful.
>
>
>
> Log =
>
>
>
> INFO [2017-12-08 12:21:36,847] ({main}
> ZeppelinConfiguration.java[create]:101) - Load configuration from
> file:/etc/zeppelin/zeppelin-site.xml
> INFO [2017-12-08 12:21:36,896] ({main}
> ZeppelinConfiguration.java[create]:109) - Server Host: 0.0.0.0
> INFO [2017-12-08 12:21:36,896] ({main}
> ZeppelinConfiguration.java[create]:113) - Server SSL Port: 8080
> INFO [2017-12-08 12:21:36,896] ({main}
> ZeppelinConfiguration.java[create]:115) - Context Path: /
> INFO [2017-12-08 12:21:36,900] ({main}
> ZeppelinConfiguration.java[create]:116) - Zeppelin Version: 0.7.3
> INFO [2017-12-08 12:21:36,917] ({main} Log.java[initialized]:186) -
> Logging initialized @275ms
> INFO [2017-12-08 12:21:36,979] ({main}
> ZeppelinServer.java[setupWebAppContext]:346) - ZeppelinServer Webapp path:
> /opt/zeppelin/webapps
> INFO [2017-12-08 12:21:37,196] ({main}
> IniRealm.java[processDefinitions]:188) - IniRealm defined, but there is no
> [users] section defined. This realm will not be populated with any users
> and it is assumed that they will be populated programatically. Users must
> be defined for this Realm instance to be useful.
> INFO [2017-12-08 12:21:37,197] ({main}
> AuthorizingRealm.java[getAuthorizationCacheLazy]:248) - No cache or
> cacheManager properties have been set. Authorization cache cannot be
> obtained.
> INFO [2017-12-08 12:21:37,227] ({main} ZeppelinServer.java[main]:187) -
> Starting zeppelin server
> INFO [2017-12-08 12:21:37,229] ({main} Server.java[doStart]:327) -
> jetty-9.2.15.v20160210
> INFO [2017-12-08 12:21:38,913] ({main}
> StandardDescriptorProcessor.java[visitServlet]:297) - NO JSP Support for /,
> did not find org.eclipse.jetty.jsp.JettyJspServlet
> INFO [2017-12-08 12:21:38,922] ({main} ContextHandler.java[log]:2052) -
> Initializing Shiro environment
> INFO [2017-12-08 12:21:38,922] ({main}
> EnvironmentLoader.java[initEnvironment]:128) - Starting Shiro environment
> initialization.
> INFO [2017-12-08 12:21:38,972] ({main}
> IniRealm.java[processDefinitions]:188) - IniRealm defined, but there is no
> [users] section defined. This realm will not be populated with any users
> and it is assumed that they will be populated programatically. Users must
> be defined for this Realm instance to be useful.
> INFO [2017-12-08 12:21:38,972] ({main}
> AuthorizingRealm.java[getAuthorizationCacheLazy]:248) - No cache or
> cacheManager properties have been set. Authorization cache cannot be
> obtained.
> INFO [2017-12-08 12:21:38,977] ({main}
> EnvironmentLoader.java[initEnvironment]:141) - Shiro environment
> initialized in 55 ms.
> WARN [2017-12-08 12:21:39,115] ({main} Helium.java[loadConf]:101) -
> /etc/zeppelin/helium.json does not exists
> WARN [2017-12-08 12:21:39,402] ({main} Interpreter.java[register]:406) -
> Static initialization is deprecated for interpreter sql, You should change
> it to use interpreter-setting.json in your jar or
> interpreter/{interpreter}/interpreter-setting.json
> INFO [2017-12-08 12:21:39,403] ({main}
> InterpreterSettingManager.java[init]:305) - Interpreter psql.sql found.
> class=org.apache.zeppelin.postgresql.PostgreSqlInterpreter
> INFO [2017-12-08 12:21:39,497] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name
> ignite
> INFO [2017-12-08 12:21:39,497] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name
> python
> INFO [2017-12-08 12:21:39,497] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name jdbc
> INFO [2017-12-08 12:21:39,497] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name psql
> INFO [2017-12-08 12:21:39,497] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name lens
> INFO [2017-12-08 12:21:39,497] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name pig
>
> INFO [2017-12-08 12:21:39,498] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name flink
> INFO [2017-12-08 12:21:39,498] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name
> angular
> INFO [2017-12-08 12:21:39,498] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name livy
> INFO [2017-12-08 12:21:39,498] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name file
> INFO [2017-12-08 12:21:39,498] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name
> elasticsearch
> INFO [2017-12-08 12:21:39,498] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name
> cassandra
> INFO [2017-12-08 12:21:39,498] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name sh
> INFO [2017-12-08 12:21:39,498] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name spark
> INFO [2017-12-08 12:21:39,498] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name md
> INFO [2017-12-08 12:21:39,498] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name
> alluxio
> INFO [2017-12-08 12:21:39,499] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name
> bigquery
> INFO [2017-12-08 12:21:39,499] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name hbase
> INFO [2017-12-08 12:21:39,499] ({main}
> InterpreterSettingManager.java[init]:337) - InterpreterSettingRef name kylin
> INFO [2017-12-08 12:21:39,532] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group md :
> id=2D18QE9AX, name=md
> INFO [2017-12-08 12:21:39,533] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group flink
> : id=2CYSDG6AU, name=flink
> INFO [2017-12-08 12:21:39,533] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group
> angular : id=2CZY6QWUG, name=angular
> INFO [2017-12-08 12:21:39,533] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group sh :
> id=2CYZFQZXG, name=sh
> INFO [2017-12-08 12:21:39,533] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group file :
> id=2D1RRAN3P, name=file
> INFO [2017-12-08 12:21:39,533] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group python
> : id=2D1A42TEJ, name=python
> INFO [2017-12-08 12:21:39,533] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group livy :
> id=2D26DBPPT, name=livy
> INFO [2017-12-08 12:21:39,533] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group psql :
> id=2CYQH5RKE, name=psql
> INFO [2017-12-08 12:21:39,533] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group kylin
> : id=2D1VSHNAX, name=kylin
> INFO [2017-12-08 12:21:39,533] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group lens :
> id=2D3AQXBDD, name=lens
> INFO [2017-12-08 12:21:39,533] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group jdbc :
> id=2CZ1RS873, name=jdbc
> INFO [2017-12-08 12:21:39,534] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group
> cassandra : id=2D338M3RA, name=cassandra
> INFO [2017-12-08 12:21:39,534] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group
> elasticsearch : id=2D372MTWM, name=elasticsearch
> INFO [2017-12-08 12:21:39,534] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group ignite
> : id=2D3A12WV4, name=ignite
> INFO [2017-12-08 12:21:39,534] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group
> alluxio : id=2D1D8DBB6, name=alluxio
> INFO [2017-12-08 12:21:39,534] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group hbase
> : id=2CZ72SGDR, name=hbase
> INFO [2017-12-08 12:21:39,534] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group pig :
> id=2CZJA495Z, name=pig
> INFO [2017-12-08 12:21:39,534] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group spark
> : id=2CYVF45A9, name=spark
> INFO [2017-12-08 12:21:39,534] ({main}
> InterpreterSettingManager.java[init]:366) - InterpreterSetting group
> bigquery : id=2D1KTF7YE, name=bigquery
> INFO [2017-12-08 12:21:39,547] ({main}
> InterpreterFactory.java[<init>]:130) - shiroEnabled: false
> INFO [2017-12-08 12:21:39,587] ({main} VfsLog.java[info]:138) - Using
> "/tmp/vfs_cache" as temporary files store.
> INFO [2017-12-08 12:21:39,621] ({main} GitNotebookRepo.java[<init>]:63) -
> Opening a git repo at '/opt/zeppelin/notebook'
> INFO [2017-12-08 12:21:39,840] ({main}
> NotebookAuthorization.java[loadFromFile]:96) -
> /etc/zeppelin/notebook-authorization.json
> INFO [2017-12-08 12:21:39,843] ({main} Credentials.java[loadFromFile]:102)
> - /etc/zeppelin/credentials.json
> INFO [2017-12-08 12:21:39,867] ({main}
> StdSchedulerFactory.java[instantiate]:1184) - Using default implementation
> for ThreadExecutor
> INFO [2017-12-08 12:21:39,870] ({main}
> SimpleThreadPool.java[initialize]:268) - Job execution threads will use
> class loader of thread: main
> INFO [2017-12-08 12:21:39,880] ({main}
> SchedulerSignalerImpl.java[<init>]:61) - Initialized Scheduler Signaller of
> type: class org.quartz.core.SchedulerSignalerImpl
> INFO [2017-12-08 12:21:39,881] ({main} QuartzScheduler.java[<init>]:240) -
> Quartz Scheduler v.2.2.1 created.
> INFO [2017-12-08 12:21:39,881] ({main} RAMJobStore.java[initialize]:155) -
> RAMJobStore initialized.
> INFO [2017-12-08 12:21:39,882] ({main}
> QuartzScheduler.java[initialize]:305) - Scheduler meta-data: Quartz
> Scheduler (v2.2.1) 'DefaultQuartzScheduler' with instanceId 'NON_CLUSTERED'
> Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
> NOT STARTED.
> Currently in standby mode.
> Number of jobs executed: 0
> Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
> Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support
> persistence. and is not clustered.
> INFO [2017-12-08 12:21:39,882] ({main}
> StdSchedulerFactory.java[instantiate]:1339) - Quartz scheduler
> 'DefaultQuartzScheduler' initialized from default resource file in Quartz
> package: 'quartz.properties'
> INFO [2017-12-08 12:21:39,882] ({main}
> StdSchedulerFactory.java[instantiate]:1343) - Quartz scheduler version:
> 2.2.1
> INFO [2017-12-08 12:21:39,882] ({main} QuartzScheduler.java[start]:575) -
> Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED started.
> INFO [2017-12-08 12:21:39,982] ({main} FolderView.java[createFolder]:107)
> - Create folder Zeppelin Tutorial
> INFO [2017-12-08 12:21:39,982] ({main} FolderView.java[createFolder]:107)
> - Create folder /
> INFO [2017-12-08 12:21:39,983] ({main} Folder.java[setParent]:168) - Set
> parent of / to /
> INFO [2017-12-08 12:21:39,983] ({main} Folder.java[setParent]:168) - Set
> parent of Zeppelin Tutorial to /
> INFO [2017-12-08 12:21:39,983] ({main} Folder.java[addNote]:184) - Add
> note 2C2AUG798 to folder Zeppelin Tutorial
> INFO [2017-12-08 12:21:39,994] ({main} Folder.java[addNote]:184) - Add
> note 2BWJFTXKJ to folder Zeppelin Tutorial
> INFO [2017-12-08 12:21:40,000] ({main} Folder.java[addNote]:184) - Add
> note 2C35YU814 to folder Zeppelin Tutorial
> INFO [2017-12-08 12:21:40,002] ({main} Folder.java[addNote]:184) - Add
> note 2A94M5J1Z to folder Zeppelin Tutorial
> INFO [2017-12-08 12:21:40,009] ({main} Folder.java[addNote]:184) - Add
> note 2BYEZ5EVK to folder Zeppelin Tutorial
> INFO [2017-12-08 12:21:40,011] ({main} Folder.java[addNote]:184) - Add
> note 2C57UKYWR to folder Zeppelin Tutorial
> INFO [2017-12-08 12:21:40,011] ({main} Notebook.java[<init>]:127) -
> Notebook indexing started...
> INFO [2017-12-08 12:21:40,142] ({main}
> LuceneSearch.java[addIndexDocs]:305) - Indexing 6 notebooks took 130ms
> INFO [2017-12-08 12:21:40,142] ({main} Notebook.java[<init>]:129) -
> Notebook indexing finished: 6 indexed in 0s
> INFO [2017-12-08 12:21:40,245] ({main}
> ServerImpl.java[initDestination]:94) - Setting the server's publish address
> to be /
> INFO [2017-12-08 12:21:40,688] ({main} ContextHandler.java[doStart]:744) -
> Started o.e.j.w.WebAppContext@418e7838
> {/,file:/opt/zeppelin-0.7.3/webapps/webapp/,AVAILABLE}{/opt/zeppelin/zeppelin-web-0.7.3.war}
> INFO [2017-12-08 12:21:40,743] ({main}
> AbstractConnector.java[doStart]:266) - Started ServerConnector@2630dbc4
> {SSL-HTTP/1.1}{0.0.0.0:8080}
> INFO [2017-12-08 12:21:40,743] ({main} Server.java[doStart]:379) - Started
> @4103ms
> INFO [2017-12-08 12:21:40,743] ({main} ZeppelinServer.java[main]:197) -
> Done, zeppelin server started
> INFO [2017-12-08 12:21:42,392] ({qtp1632392469-71}
> NotebookServer.java[onOpen]:157) - New connection from 10.136.169.200 :
> 43470
> INFO [2017-12-08 12:21:46,800] ({qtp1632392469-71}
> AbstractValidatingSessionManager.java[enableSessionValidation]:230) -
> Enabling session validation scheduler...
> INFO [2017-12-08 12:21:46,864] ({qtp1632392469-71}
> AuthorizingRealm.java[getAuthorizationCacheLazy]:248) - No cache or
> cacheManager properties have been set. Authorization cache cannot be
> obtained.
> INFO [2017-12-08 12:21:46,896] ({qtp1632392469-71}
> AuthorizingRealm.java[getAuthorizationCacheLazy]:248) - No cache or
> cacheManager properties have been set. Authorization cache cannot be
> obtained.
> INFO [2017-12-08 12:21:46,931] ({qtp1632392469-71}
> AuthorizingRealm.java[getAuthorizationCacheLazy]:248) - No cache or
> cacheManager properties have been set. Authorization cache cannot be
> obtained.
> INFO [2017-12-08 12:21:46,958] ({qtp1632392469-71}
> AuthorizingRealm.java[getAuthorizationCacheLazy]:248) - No cache or
> cacheManager properties have been set. Authorization cache cannot be
> obtained.
> INFO [2017-12-08 12:21:46,992] ({qtp1632392469-71}
> AuthorizingRealm.java[getAuthorizationCacheLazy]:248) - No cache or
> cacheManager properties have been set. Authorization cache cannot be
> obtained.
> INFO [2017-12-08 12:21:47,019] ({qtp1632392469-71}
> AuthorizingRealm.java[getAuthorizationCacheLazy]:248) - No cache or
> cacheManager properties have been set. Authorization cache cannot be
> obtained.
> INFO [2017-12-08 12:21:47,047] ({qtp1632392469-71}
> AuthorizingRealm.java[getAuthorizationCacheLazy]:248) - No cache or
> cacheManager properties have been set. Authorization cache cannot be
> obtained.
> INFO [2017-12-08 12:21:47,076] ({qtp1632392469-71}
> AuthorizingRealm.java[getAuthorizationCacheLazy]:248) - No cache or
> cacheManager properties have been set. Authorization cache cannot be
> obtained.
> WARN [2017-12-08 12:21:47,109] ({qtp1632392469-71}
> LoginRestApi.java[postLogin]:119) -
> {"status":"OK","message":"","body":{"principal":"zepptest","ticket":"e0c65401-e791-4fd9-bcc0-ec68b47f2b27","roles":"[ipausers]"}}
> INFO [2017-12-08 12:22:03,273] ({qtp1632392469-71}
> AuthorizingRealm.java[getAuthorizationCacheLazy]:248) - No cache or
> cacheManager properties have been set. Authorization cache cannot be
> obtained.
> INFO [2017-12-08 12:22:03,290] ({qtp1632392469-77}
> AuthorizingRealm.java[getAuthorizationCacheLazy]:248) - No cache or
> cacheManager properties have been set. Authorization cache cannot be
> obtained.
> INFO [2017-12-08 12:22:03,305] ({qtp1632392469-79}
> AuthorizingRealm.java[getAuthorizationCacheLazy]:248) - No cache or
> cacheManager properties have been set. Authorization cache cannot be
> obtained.
>
> OUT
>
> Dec 08, 2017 12:20:03 PM com.sun.jersey.api.core.ScanningResourceConfig
> logClasses
> INFO: Root resource classes found:
> class org.apache.zeppelin.rest.ConfigurationsRestApi
> class org.apache.zeppelin.rest.InterpreterRestApi
> class org.apache.zeppelin.rest.CredentialRestApi
> class org.apache.zeppelin.rest.LoginRestApi
> class org.apache.zeppelin.rest.NotebookRestApi
> class org.apache.zeppelin.rest.NotebookRepoRestApi
> class org.apache.zeppelin.rest.SecurityRestApi
> class org.apache.zeppelin.rest.ZeppelinRestApi
> class org.apache.zeppelin.rest.HeliumRestApi
> Dec 08, 2017 12:20:03 PM com.sun.jersey.api.core.ScanningResourceConfig
> init
> INFO: No provider classes found.
> Dec 08, 2017 12:20:03 PM
> com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
> INFO: Initiating Jersey application, version 'Jersey: 1.13 06/29/2012
> 05:14 PM'
> Dec 08, 2017 12:20:03 PM com.sun.jersey.spi.inject.Errors
> processErrorMessages
> WARNING: The following warnings have been detected with resource and/or
> provider classes:
> WARNING: A HTTP GET method, public javax.ws.rs.core.Response
> org.apache.zeppelin.rest.CredentialRestApi.getCredentials(java.lang.String)
> throws java.io.IOException,java.lang.IllegalArgumentException, should not
> consume any entity.
> WARNING: A HTTP GET method, public javax.ws.rs.core.Response
> org.apache.zeppelin.rest.InterpreterRestApi.listInterpreter(java.lang.String),
> should not consume any entity.
> WARNING: A sub-resource method, public javax.ws.rs.core.Response
> org.apache.zeppelin.rest.NotebookRestApi.createNote(java.lang.String)
> throws java.io.IOException, with URI template, "/", is treated as a
> resource method
> WARNING: A sub-resource method, public javax.ws.rs.core.Response
> org.apache.zeppelin.rest.NotebookRestApi.getNoteList() throws
> java.io.IOException, with URI template, "/", is treated as a resource method
> ZEPPELIN_CLASSPATH:
> ::/opt/zeppelin/lib/interpreter/*:/opt/zeppelin/lib/*:/opt/zeppelin/*::/etc/zeppelin
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option
> MaxPermSize=512m; support was removed in 8.0
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/opt/zeppelin-0.7.3/lib/interpreter/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/zeppelin-0.7.3/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> Dec 08, 2017 12:20:37 PM com.sun.jersey.api.core.PackagesResourceConfig
> init
> INFO: Scanning for root resource and provider classes in the packages:
> org.apache.zeppelin.rest
>
> *De :* Jeff Zhang [mailto:zjf...@gmail.com]
> *Envoyé :* mardi 6 février 2018 11:17
>
>
> *À :* users@zeppelin.apache.org
> *Objet :* Re: How to
>
> have a native graphical representation (%sql) of a HiveContext?
>
>
>
>
>
> Could you attach the spark interpreter log ?
>
>
>
> LINZ, Arnaud <al...@bouyguestelecom.fr>于2018年2月6日周二 下午4:49写道：
>
> I really tried, but it really does not work…
>
> - zeppelin.spark.useHiveContext is set to true
> - I directly use sqlContext without creating it
> - I have a link to hive-site.xml in /etc/spark/conf
>
> But sqlContext does not see any hive table
>
>
>
> I don’t see any error log, or any helpful log in zeppelin logs. What can
> be wrong?
>
>
>
>
>
> *De :* Jeff Zhang [mailto:zjf...@gmail.com]
>
> *Envoyé :* lundi 5 février 2018 14:31
>
>
> *À :* users@zeppelin.apache.org
> *Objet :* Re: How to have a native graphical representation (%sql) of a
> HiveContext?
>
>
>
> Sorry I should be more accurate.
>
> Just don't create SqlContext/HiveContext in zeppelin as zeppelin will
> create that for you. If you zeppelin.spark.useHiveContext to true variable
> sqlContext
> will be HiveContext, other it is SqlContxt
>
>
>
> LINZ, Arnaud <al...@bouyguestelecom.fr>于2018年2月5日周一 下午9:15写道：
>
> Thanks for your answer, but it does not address my problem.
>
> - I don’t create sqlContext, I use the one provided by Zeppelin. But
> sqlContext is not a hive Context and cannot access hive metastore.
> - Zeppelin can see my hive conf files, and selecting tables through a
> created hiveContext works. But I cannot visualize them in the %sql
> graphical interpretor.
>
>
>
> *De :* Jeff Zhang [mailto:zjf...@gmail.com]
> *Envoyé :* lundi 5 février 2018 14:01
> *À :* users@zeppelin.apache.org
> *Objet :* Re: How to have a native graphical representation (%sql) of a
> HiveContext?
>
>
>
>
>
> 1. Don't create sqlContext in zeppelin as zeppelin will create that for
> you, and %sql use the sqlContext created by zeppelin itself.
>
> 2. Make sure you have hive-site.xml under SPARK_CONF_DIR if you want to
> use hiveContext. Otherwise spark will use single user derby instance which
> is not for production, and will cause conflicts when you create multiple
> spark interpreter in one zeppelin instance.
>
>
>
>
>
> LINZ, Arnaud <al...@bouyguestelecom.fr>于2018年2月5日周一 下午8:33写道：
>
> Hello,
>
>
>
> I’m trying to install Zeppelin (0.7.2) on my CDH cluster, and I am unable
> to connect the sql + graphical representations of the %sql interpreter
> with my Hive data, and more surprisingly I really can’t find any good
> source on the internet (apache zeppelin documentation or stack overflow)
> that gives a practical answer about how to do this.
>
> Most of the time, the data comes from compressed Hive tables and not plain
> hdfs text files ; so using a hive context is far more convenient than a
> plain spark sql context.
>
>
>
> The following :
>
> %spark
>
> val hc = new org.apache.spark.sql.hive.HiveContext(sc)
>
> val result = hc.sql("select * from hivedb.hivetable")
>
> result.registerTempTable("myTest")
>
>
>
> works but no myTest table is available in the following %sql interpreter :
>
> %sql
>
> select * from myTest
>
> org.apache.spark.sql.AnalysisException: Table not found: myTest;
>
>
>
>
>
> However the following :
>
> %pyspark
>
> result = sqlContext.read.text("hdfs://cluster/test.txt")
>
> result.registerTempTable("mySqlTest")
>
>
>
> works as the %sql interpreter is “plugged” to the sqlContext
>
>
>
> but
>
> result = sqlContext.sql("select * from hivedb.hivetable") does not work
> as the sqlContext is not a hive context.
>
>
>
> I have set zeppelin.spark.useHiveContext to true, but it seems to have no
> effect (btw, it was more of a wild guess since the documentation is not
> giving much detail on parameters and context configuration)
>
>
>
> Can you direct me towards how to configure the context used by the %sql
> interpreter?
>
>
>
> Best regards,
>
> Arnaud
>
>
>
> PS : %spark and %sql interpreter conf:
>
>
>
> master yarn-client
>
> spark.app.name Zeppelin
>
> spark.cores.max
>
> spark.executor.memory 5g
>
> zeppelin.R.cmd R
>
> zeppelin.R.image.width 100%
>
> zeppelin.R.knitr true
>
> zeppelin.R.render.options out.format = 'html', comment = NA, echo =
> FALSE, results = 'asis', message = F, warning = F
>
> zeppelin.dep.additionalRemoteRepository spark-packages,
> http://dl.bintray.com/spark-packages/maven,false;
>
> zeppelin.dep.localrepo local-repo
>
> zeppelin.interpreter.localRepo /opt/zeppelin/local-repo/2CYVF45A9
>
> zeppelin.interpreter.output.limit 102400
>
> zeppelin.pyspark.python /usr/bin/pyspark
>
> zeppelin.spark.concurrentSQL true
>
> zeppelin.spark.importImplicit true
>
> zeppelin.spark.maxResult 1000
>
> zeppelin.spark.printREPLOutput true
>
> zeppelin.spark.sql.stacktrace true
>
> zeppelin.spark.useHiveContext true
>
>
> ------------------------------
>
>
> L'intégrité de ce message n'étant pas assurée sur internet, la société
> expéditrice ne peut être tenue responsable de son contenu ni de ses pièces
> jointes. Toute utilisation ou diffusion non autorisée est interdite. Si
> vous n'êtes pas destinataire de ce message, merci de le détruire et
> d'avertir l'expéditeur.
>
> The integrity of this message cannot be guaranteed on the Internet. The
> company that sent this message cannot therefore be held liable for its
> content nor attachments. Any unauthorized use or dissemination is
> prohibited. If you are not the intended recipient of this message, then
> please delete it and notify the sender.
>
>
