I am facing a similar issue while using Spark R.
My environment:
- Spark 2.0.2
- Zeppelin: 0.6.2
- Java 1.8.0_111
- R: 3.3.1
SPARK_HOME is set. I am trying to run a simple paragraph.
%r
print("hello ...")
I get the following exception
*Interpreter Log*
> # getZeppelinR
> .zeppelinR = SparkR:::callJStatic("org.apache.zeppelin.spark.ZeppelinR",
"getZeppelinR", hashCode)
at org.apache.zeppelin.spark.ZeppelinR.waitForRScriptInitialized(
ZeppelinR.java:295)
at org.apache.zeppelin.spark.ZeppelinR.request(ZeppelinR.java:235)
at org.apache.zeppelin.spark.ZeppelinR.eval(ZeppelinR.java:183)
at org.apache.zeppelin.spark.ZeppelinR.open(ZeppelinR.java:172)
at org.apache.zeppelin.spark.SparkRInterpreter.open(
SparkRInterpreter.java:85)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(
LazyOpenInterpreter.java:69)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgress(
LazyOpenInterpreter.java:110)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.
getProgress(RemoteInterpreterServer.java:404)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
Processor$getProgress.getResult(RemoteInterpreterService.java:1509)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
Processor$getProgress.getResult(RemoteInterpreterService.java:1494)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(
TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
*Zeppelin Log*
ERROR [2016-11-16 14:42:05,664] ({Thread-377} JobProgressPoller.java[run]:54)
- Can not get or update progress
org.apache.zeppelin.interpreter.InterpreterException:
org.apache.thrift.transport.TTransportException
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getProgress(
RemoteInterpreter.java:373)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgress(
LazyOpenInterpreter.java:111)
at org.apache.zeppelin.notebook.Paragraph.progress(Paragraph.java:237)
at org.apache.zeppelin.scheduler.JobProgressPoller.run(
JobProgressPoller.java:51)
Caused by: org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TIOStreamTransport.read(
TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(
TBinaryProtocol.java:429)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(
TBinaryProtocol.java:318)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
TBinaryProtocol.java:219)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
Client.recv_getProgress(RemoteInterpreterService.java:296)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
Client.getProgress(RemoteInterpreterService.java:281)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getProgress(
RemoteInterpreter.java:370)
... 3 more
When I tried run R commands using sparkR by launching
$SPARK_HOME/bin/sparkR, it works.
Would much appreciate any help.
Thanks!
Abul
On Fri, Sep 16, 2016 at 5:36 PM, Hyung Sung Shim <[email protected]> wrote:
> Hello.
> I think that `zeppelin.interpreter.host` and `zeppelin.interpreter.port`
> values on Interpreter setting UI should be removed because Interpreter
> will automatically be launched when you run a paragraph.
>
> 2016-09-16 3:56 GMT+09:00 Arpan Saha <[email protected]>:
>
>> Yes. It is a zeppelin log. Not spark interpreter log. Can you please tell
>> me where can I get spark interpreter log? I don't find anything under
>> zeppelin/logs/ path.
>>
>> Thanks,
>> Arpan
>> ________________________________________
>> From: DuyHai Doan [[email protected]]
>> Sent: Thursday, September 15, 2016 11:20 AM
>> To: [email protected]
>> Subject: Re: FW: Issue with Zeppelin setup on Datastax-Spark
>>
>> Is it Zeppelin log or Spark interpreter log ? I guess we need also the
>> spark interpreter log to understand what is breaking Spark under the hood.
>> org.apache.thrift.transport.TTransportException is just technical
>> exception on inter-jvm communication
>>
>> On Thu, Sep 15, 2016 at 7:13 PM, Arpan Saha <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>> Hi,
>>
>> Please find below full stack trace.
>>
>> ERROR [2016-09-15 16:49:15,117] ({pool-1-thread-2}
>> RemoteInterpreter.java[init]:182) - Failed to create interpreter:
>> org.apache.zeppelin.spark.SparkInterpreter
>> ERROR [2016-09-15 16:49:15,118] ({pool-1-thread-2} Job.java[run]:189) -
>> Job failed
>> org.apache.zeppelin.interpreter.InterpreterException:
>> org.apache.thrift.transport.TTransportException
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.ini
>> t(RemoteInterpreter.java:183)
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>> FormType(RemoteInterpreter.java:328)
>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormT
>> ype(LazyOpenInterpreter.java:105)
>> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java
>> :260)
>> at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
>> at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(
>> RemoteScheduler.java:328)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executor
>> s.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>> tureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>> tureTask.run(ScheduledThreadPoolExecutor.java:293)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>> Executor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>> lExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: org.apache.thrift.transport.TTransportException
>> at org.apache.thrift.transport.TIOStreamTransport.read(TIOStrea
>> mTransport.java:132)
>> at org.apache.thrift.transport.TTransport.readAll(TTransport.ja
>> va:86)
>> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryPr
>> otocol.java:429)
>> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryPr
>> otocol.java:318)
>> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
>> TBinaryProtocol.java:219)
>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.
>> java:69)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Client.recv_createInterpreter(RemoteInterpreterService.java:184)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Client.createInterpreter(RemoteInterpreterService.java:168)
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.ini
>> t(RemoteInterpreter.java:172)
>> ... 12 more
>> ERROR [2016-09-15 16:49:15,118] ({Thread-18}
>> RemoteInterpreterEventPoller.java[run]:88) - Can't get
>> RemoteInterpreterEvent
>>
>> Thanks,
>> Arpan.
>>
>> ________________________________________
>> From: DuyHai Doan [[email protected]<mailto:[email protected]>]
>> Sent: Thursday, September 15, 2016 4:29 AM
>> To: [email protected]<mailto:[email protected]>
>> Subject: Re: Issue with Zeppelin setup on Datastax-Spark
>>
>> Can you provide the complete stack trace ?
>>
>> Also, worth reading my blog post about integrating Spark/Zeppelin/C* or
>> DSE : http://www.doanduyhai.com/blog/?p=2325
>>
>> On Thu, Sep 15, 2016 at 5:17 AM, Arpan Saha <[email protected]
>> <mailto:[email protected]><mailto:[email protected]<mailto:
>> [email protected]>>> wrote:
>> Hi Team,
>>
>> I am trying to integrate Zeppelin 0.6.0 with DataStax 4.8.8 (which has
>> Spark 1.4.2). After I configured following properties in zeppelin-env.sh
>> when I start zeppelin daemon it started and in the browser I can see
>> zeppelin is running but when I am trying to execute spark query in the
>> notebook it is throwing below Error. Could you please help me to solve this
>> issue.
>>
>> export JAVA_HOME=<JAVA Home Path>
>> export SPARK_HOME=/etc/dse/spark
>> export HADOOP_CONF_DIR=/etc/dse/hadoop
>> export MASTER=spark://<masterhost>:7077
>>
>> Also, added/updated below properties in the Spark Interpreter screen of
>> Zeppelin UI.
>>
>> Master
>> spark.app.name<http://spark.app.name><http://spark.app.name>
>> spark.cassandra.auth.password
>> spark.cassandra.auth.username
>> spark.cassandra.connection.host
>> spark.cores.max
>> spark.executor.memory
>> zeppelin.interpreter.host
>> zeppelin.interpreter.port
>>
>> Trying to execute below statement in the notebook:
>> %spark
>> sc.version
>>
>> Below Error is getting:
>> java.net.SocketException: Connection reset at
>> java.net.SocketInputStream.read(SocketInputStream.java:209) at
>> java.net.SocketInputStream.read(SocketInputStream.java:141) at
>> java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at
>> java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at
>> java.io.BufferedInputStream.read(BufferedInputStream.java:345) at
>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) at
>> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
>> at
>> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
>> at
>> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Client.recv_createInterpreter(RemoteInterpreterService.java:184) at
>> org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Client.createInterpreter(RemoteInterpreterService.java:168) at
>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:172)
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>> FormType(RemoteInterpreter.java:328) at org.apache.zeppelin.interprete
>> r.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
>>
>> [email protected]<mailto:users-subscribe@z
>> eppelin.apache.org><mailto:[email protected]<mailto:
>> [email protected]>>
>>
>> Thanks,
>> Arpan.
>>
>> ________________________________
>> Notice: This e-mail message has been sent by an employee or contractor of
>> American Express Global Business Travel (“GBT”), a joint venture that is
>> not wholly-owned by American Express Company or any of its subsidiaries
>> (“American Express”). E-mail from GBT employees and contractors may
>> continue to utilize an American Express domain for some period; however
>> American Express is not responsible for the content of this email, which is
>> controlled by GBT. “American Express Global Business Travel”, “American
>> Express” and the American Express Logo are trademarks of American Express
>> and are used by GBT under limited license.
>> ________________________________
>> Important : Cet email a été envoyé par un employé ou un sous-traitant
>> d’American Express Global Business Travel (« GBT »), une joint-venture qui
>> n’est pas intégralement détenue par American Express Company ou l’une de
>> ses filiales (« American Express »). Les emails des employés et
>> sous-traitants de GBT peuvent continuer de comporter un nom de domaine
>> d’American Express pour une certaine période ; toutefois, American Express
>> n’est pas responsable du contenu de cet email, qui est contrôlé par GBT. «
>> American Express Global Business Travel », « American Express » et le logo
>> American Express sont des marques d’American Express et sont utilisées par
>> GBT aux termes d’une licence limitée.
>>
>> ________________________________
>>
>> http://www.mindtree.com/email/disclaimer.html
>>
>>
>>
>
