I recommend downloading my pre-built Zeppelin for Datastax. Shared folder
link: https://drive.google.com/folderview?id=0B6wR2aj4Cb6wQ01aR3ItR0xUNms
On Wed, Nov 16, 2016 at 11:13 AM, DuyHai Doan <doanduy...@gmail.com> wrote:
> Ok I understand why you have issue.
>
> You are using Spark 2.0.2 and latest Datastax 5.0.3 is still using Spark
> version 1.6.X
>
>
>
> On Wed, Nov 16, 2016 at 10:23 AM, Abul Basar <aba...@einext.com> wrote:
>
>> I am facing a similar issue while using Spark R.
>>
>> My environment:
>>
>> - Spark 2.0.2
>> - Zeppelin: 0.6.2
>> - Java 1.8.0_111
>> - R: 3.3.1
>>
>>
>> SPARK_HOME is set. I am trying to run a simple paragraph.
>>
>> %r
>> print("hello ...")
>>
>> I get the following exception
>>
>> *Interpreter Log*
>> > # getZeppelinR
>>
>> > .zeppelinR = SparkR:::callJStatic("org.apache.zeppelin.spark.ZeppelinR",
>> "getZeppelinR", hashCode)
>>
>> at org.apache.zeppelin.spark.ZeppelinR.waitForRScriptInitialize
>> d(ZeppelinR.java:295)
>> at org.apache.zeppelin.spark.ZeppelinR.request(ZeppelinR.java:235)
>> at org.apache.zeppelin.spark.ZeppelinR.eval(ZeppelinR.java:183)
>> at org.apache.zeppelin.spark.ZeppelinR.open(ZeppelinR.java:172)
>> at org.apache.zeppelin.spark.SparkRInterpreter.open(SparkRInter
>> preter.java:85)
>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(Laz
>> yOpenInterpreter.java:69)
>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgr
>> ess(LazyOpenInterpreter.java:110)
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServ
>> er.getProgress(RemoteInterpreterServer.java:404)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Processor$getProgress.getResult(RemoteInterpreterService.java:1509)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Processor$getProgress.getResult(RemoteInterpreterService.java:1494)
>> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run
>> (TThreadPoolServer.java:285)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>> Executor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>> lExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>>
>>
>> *Zeppelin Log*
>> ERROR [2016-11-16 14:42:05,664] ({Thread-377}
>> JobProgressPoller.java[run]:54) - Can not get or update progress
>> org.apache.zeppelin.interpreter.InterpreterException:
>> org.apache.thrift.transport.TTransportException
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>> Progress(RemoteInterpreter.java:373)
>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgr
>> ess(LazyOpenInterpreter.java:111)
>> at org.apache.zeppelin.notebook.Paragraph.progress(Paragraph.java:237)
>> at org.apache.zeppelin.scheduler.JobProgressPoller.run(JobProgr
>> essPoller.java:51)
>> Caused by: org.apache.thrift.transport.TTransportException
>> at org.apache.thrift.transport.TIOStreamTransport.read(TIOStrea
>> mTransport.java:132)
>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
>> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryPr
>> otocol.java:429)
>> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryPr
>> otocol.java:318)
>> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
>> TBinaryProtocol.java:219)
>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Client.recv_getProgress(RemoteInterpreterService.java:296)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Client.getProgress(RemoteInterpreterService.java:281)
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>> Progress(RemoteInterpreter.java:370)
>> ... 3 more
>>
>>
>>
>> When I tried run R commands using sparkR by launching
>> $SPARK_HOME/bin/sparkR, it works.
>>
>>
>> Would much appreciate any help.
>>
>> Thanks!
>> Abul
>>
>>
>> On Fri, Sep 16, 2016 at 5:36 PM, Hyung Sung Shim <hss...@nflabs.com>
>> wrote:
>>
>>> Hello.
>>> I think that `zeppelin.interpreter.host` and `zeppelin.interpreter.port`
>>> values on Interpreter setting UI should be removed because Interpreter
>>> will automatically be launched when you run a paragraph.
>>>
>>> 2016-09-16 3:56 GMT+09:00 Arpan Saha <arpan.s...@mindtree.com>:
>>>
>>>> Yes. It is a zeppelin log. Not spark interpreter log. Can you please
>>>> tell me where can I get spark interpreter log? I don't find anything under
>>>> zeppelin/logs/ path.
>>>>
>>>> Thanks,
>>>> Arpan
>>>> ________________________________________
>>>> From: DuyHai Doan [doanduy...@gmail.com]
>>>> Sent: Thursday, September 15, 2016 11:20 AM
>>>> To: users@zeppelin.apache.org
>>>> Subject: Re: FW: Issue with Zeppelin setup on Datastax-Spark
>>>>
>>>> Is it Zeppelin log or Spark interpreter log ? I guess we need also the
>>>> spark interpreter log to understand what is breaking Spark under the hood.
>>>> org.apache.thrift.transport.TTransportException is just technical
>>>> exception on inter-jvm communication
>>>>
>>>> On Thu, Sep 15, 2016 at 7:13 PM, Arpan Saha <arpan.s...@mindtree.com
>>>> <mailto:arpan.s...@mindtree.com>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> Please find below full stack trace.
>>>>
>>>> ERROR [2016-09-15 16:49:15,117] ({pool-1-thread-2}
>>>> RemoteInterpreter.java[init]:182) - Failed to create interpreter:
>>>> org.apache.zeppelin.spark.SparkInterpreter
>>>> ERROR [2016-09-15 16:49:15,118] ({pool-1-thread-2} Job.java[run]:189) -
>>>> Job failed
>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>> org.apache.thrift.transport.TTransportException
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.ini
>>>> t(RemoteInterpreter.java:183)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>>>> FormType(RemoteInterpreter.java:328)
>>>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormT
>>>> ype(LazyOpenInterpreter.java:105)
>>>> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java
>>>> :260)
>>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
>>>> at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(
>>>> RemoteScheduler.java:328)
>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executor
>>>> s.java:511)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>>>> tureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>>>> tureTask.run(ScheduledThreadPoolExecutor.java:293)
>>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>>>> Executor.java:1142)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>>>> lExecutor.java:617)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>> Caused by: org.apache.thrift.transport.TTransportException
>>>> at org.apache.thrift.transport.TIOStreamTransport.read(TIOStrea
>>>> mTransport.java:132)
>>>> at org.apache.thrift.transport.TTransport.readAll(TTransport.ja
>>>> va:86)
>>>> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryPr
>>>> otocol.java:429)
>>>> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryPr
>>>> otocol.java:318)
>>>> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
>>>> TBinaryProtocol.java:219)
>>>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.
>>>> java:69)
>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>> ice$Client.recv_createInterpreter(RemoteInterpreterService.java:184)
>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>> ice$Client.createInterpreter(RemoteInterpreterService.java:168)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.ini
>>>> t(RemoteInterpreter.java:172)
>>>> ... 12 more
>>>> ERROR [2016-09-15 16:49:15,118] ({Thread-18}
>>>> RemoteInterpreterEventPoller.java[run]:88) - Can't get
>>>> RemoteInterpreterEvent
>>>>
>>>> Thanks,
>>>> Arpan.
>>>>
>>>> ________________________________________
>>>> From: DuyHai Doan [doanduy...@gmail.com<mailto:doanduy...@gmail.com>]
>>>> Sent: Thursday, September 15, 2016 4:29 AM
>>>> To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>
>>>> Subject: Re: Issue with Zeppelin setup on Datastax-Spark
>>>>
>>>> Can you provide the complete stack trace ?
>>>>
>>>> Also, worth reading my blog post about integrating Spark/Zeppelin/C* or
>>>> DSE : http://www.doanduyhai.com/blog/?p=2325
>>>>
>>>> On Thu, Sep 15, 2016 at 5:17 AM, Arpan Saha <arpan.s...@mindtree.com
>>>> <mailto:arpan.s...@mindtree.com><mailto:arpan.s...@mindtree.com<mailto:
>>>> arpan.s...@mindtree.com>>> wrote:
>>>> Hi Team,
>>>>
>>>> I am trying to integrate Zeppelin 0.6.0 with DataStax 4.8.8 (which has
>>>> Spark 1.4.2). After I configured following properties in zeppelin-env.sh
>>>> when I start zeppelin daemon it started and in the browser I can see
>>>> zeppelin is running but when I am trying to execute spark query in the
>>>> notebook it is throwing below Error. Could you please help me to solve this
>>>> issue.
>>>>
>>>> export JAVA_HOME=<JAVA Home Path>
>>>> export SPARK_HOME=/etc/dse/spark
>>>> export HADOOP_CONF_DIR=/etc/dse/hadoop
>>>> export MASTER=spark://<masterhost>:7077
>>>>
>>>> Also, added/updated below properties in the Spark Interpreter screen of
>>>> Zeppelin UI.
>>>>
>>>> Master
>>>> spark.app.name<http://spark.app.name><http://spark.app.name>
>>>> spark.cassandra.auth.password
>>>> spark.cassandra.auth.username
>>>> spark.cassandra.connection.host
>>>> spark.cores.max
>>>> spark.executor.memory
>>>> zeppelin.interpreter.host
>>>> zeppelin.interpreter.port
>>>>
>>>> Trying to execute below statement in the notebook:
>>>> %spark
>>>> sc.version
>>>>
>>>> Below Error is getting:
>>>> java.net.SocketException: Connection reset at
>>>> java.net.SocketInputStream.read(SocketInputStream.java:209) at
>>>> java.net.SocketInputStream.read(SocketInputStream.java:141) at
>>>> java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at
>>>> java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at
>>>> java.io.BufferedInputStream.read(BufferedInputStream.java:345) at
>>>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
>>>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
>>>> at
>>>> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
>>>> at
>>>> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
>>>> at
>>>> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
>>>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>> ice$Client.recv_createInterpreter(RemoteInterpreterService.java:184)
>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>> ice$Client.createInterpreter(RemoteInterpreterService.java:168) at
>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:172)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>>>> FormType(RemoteInterpreter.java:328) at org.apache.zeppelin.interprete
>>>> r.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
>>>>
>>>> users-subscr...@zeppelin.apache.org<mailto:users-subscribe@z
>>>> eppelin.apache.org><mailto:users-subscr...@zeppelin.apache.org<mailto:
>>>> users-subscr...@zeppelin.apache.org>>
>>>>
>>>> Thanks,
>>>> Arpan.
>>>>
>>>> ________________________________
>>>> Notice: This e-mail message has been sent by an employee or contractor
>>>> of American Express Global Business Travel (“GBT”), a joint venture that is
>>>> not wholly-owned by American Express Company or any of its subsidiaries
>>>> (“American Express”). E-mail from GBT employees and contractors may
>>>> continue to utilize an American Express domain for some period; however
>>>> American Express is not responsible for the content of this email, which is
>>>> controlled by GBT. “American Express Global Business Travel”, “American
>>>> Express” and the American Express Logo are trademarks of American Express
>>>> and are used by GBT under limited license.
>>>> ________________________________
>>>> Important : Cet email a été envoyé par un employé ou un sous-traitant
>>>> d’American Express Global Business Travel (« GBT »), une joint-venture qui
>>>> n’est pas intégralement détenue par American Express Company ou l’une de
>>>> ses filiales (« American Express »). Les emails des employés et
>>>> sous-traitants de GBT peuvent continuer de comporter un nom de domaine
>>>> d’American Express pour une certaine période ; toutefois, American Express
>>>> n’est pas responsable du contenu de cet email, qui est contrôlé par GBT. «
>>>> American Express Global Business Travel », « American Express » et le logo
>>>> American Express sont des marques d’American Express et sont utilisées par
>>>> GBT aux termes d’une licence limitée.
>>>>
>>>> ________________________________
>>>>
>>>> http://www.mindtree.com/email/disclaimer.html
>>>>
>>>>
>>>>
>>>
>>
>
