Another more classical way to achieve this is to start the remote
interpreter via the shell command (example `./bin/interpreter.sh -p 3034 -d
markdown`).
When I configure the markdown interpreter to connect to localhost:3034, I
receive an thrift exception (*). My thrift server and client are normally
compatible.
Anyone running remote interperter with latest HEAD?
INFO [2016-09-12 15:22:38,530] ({pool-2-thread-2}
SchedulerFactory.java[jobFinished]:137) - Job
paragraph_1423836981412_-1007008116 finished by scheduler
org.apache.zeppelin.interpreter.remote.RemoteInterpreter2A94M5J1Z1515930344
ERROR [2016-09-12 15:22:38,714] ({qtp2076287037-51}
ResourcePoolUtils.java[getAllResourcesExcept]:64) - Negative length:
-2147418111
org.apache.thrift.TApplicationException: Negative length: -2147418111
at
org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
at
org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_resourcePoolGetAll(RemoteInterpreterService.java:486)
at
org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.resourcePoolGetAll(RemoteInterpreterService.java:474)
at
org.apache.zeppelin.resource.ResourcePoolUtils.getAllResourcesExcept(ResourcePoolUtils.java:58)
at
org.apache.zeppelin.resource.ResourcePoolUtils.getAllResources(ResourcePoolUtils.java:36)
at org.apache.zeppelin.helium.Helium.suggestApp(Helium.java:153)
at org.apache.zeppelin.rest.HeliumRestApi.suggest(HeliumRestApi.java:74)
at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:180)
at
org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
On Fri, Aug 19, 2016 at 6:47 PM, Sourav Mazumder <
sourav.mazumde...@gmail.com> wrote:
> Hi,
>
> From the documentation it is not very clear how exactly the remote
> interpreter works.
>
> Here is what I tried. But it didn't work. I am using zeppelin 0.6.1 binary
> and spark 2.0.
>
> 1. In a remote spark cluster
> a) I created a spark interpreter (copying from the default spark
> interpreter) with master as Yarn-Client
> b). I started the same in a notebook (ensuring that this interpreter is
> enabled) using following code -
> import org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer
>
> val interpreter = new RemoteInterpreterServer(51000)
> interpreter.start()
> c). When I go to the yarn application manager I can see this new
> interpreter is running. Also the log file shows that the remote interpreter
> has started in 51000 port
>
> 2. Next in a client machine (a mac laptop) I started Zeppelin.
> a) I created a new Spark interpreter copying from existing one. There I
> enabled the checkbox - Connect to Existing Process and provided host ip and
> port.
> b) I have kept the master as local
> c) Now I selected this interpreter for Zeppelin Tutorial notebook and ran
> the same.
> d) The first paragraph with spark interpreter (Load Data in Table) worked
> fine. Also I saw the log in the server shows that it is successfully
> parsing the bank table.
> e) But the next paragraph (%sql select age, count(1) value from bank where
> age < 30 group by age order by age) gives error as follwos -
>
> Caused by: java.lang.ClassCastException: cannot assign instance of
> scala.collection.immutable.List$SerializationProxy to field
> org.apache.spark.rdd.RDD.org$apache$spark$rdd$RDD$$dependencies_ of type
> scala.collection.Seq in instance of org.apache.spark.rdd.MapPartitionsRDD
>
> at
> java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(
> ObjectStreamClass.java:2089)
>
> at
> java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1261)
>
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1999)
>
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
>
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
>
> at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
>
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
>
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
>
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
>
> at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
>
> at java.io.ObjectInputStream.readObject(ObjectInputStream.
> java:371)
>
> at
> scala.collection.immutable.List$SerializationProxy.
> readObject(List.scala:479)
>
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:497)
>
> at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
>
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
>
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
>
> at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
>
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
>
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
>
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
>
> at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
>
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
>
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
>
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
>
> at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
>
> at java.io.ObjectInputStream.readObject(ObjectInputStream.
> java:371)
>
> at
> scala.collection.immutable.List$SerializationProxy.
> readObject(List.scala:479)
>
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:497)
>
> at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
>
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
>
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
>
> at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
>
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
>
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
>
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
>
> at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
>
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
>
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
>
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
>
> at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
>
>
> Any idea where am I going wrong ?
>
> Regards,
> Sourav
>
