AW: how to upload data files for use in notebook (python)
Thanks for the fast reply. Unfortunately, cloud storage is not the preferred option for us. We would need to host any solution on our own internal server. Von: Jeff Zhang Gesendet: Donnerstag, 1. Juli 2021 08:42 An: users Betreff: Re: how to upload data files for use in notebook (python) Unfortunately, zeppelin doesn't provide such function now. Have you consider to use cloud storage ? So that zeppelin can read from there. mailto:uwe.bau...@dlr.de>> 于2021年7月1日周四 下午2:32写道： Hi, I need to have an option to upload (csv) files with data to be processed in the notebooks (in Python). So far, I found several potential solutions: 1. Scp to the local file system on the server 2. Host the file somewhere and use wget 3. run a separate dash app on the same server for user friendly file upload (https://dash.plotly.com/dash-core-components/upload) All of these have some drawbacks and for all of them the user has to “leave” the notebook and at least for 1) also need to have appropriate credentials. Does there exist a better solution for file upload in apache zeppelin than my potential solutions above (as of today I would opt for dash)? I found https://support.labs.cognitiveclass.ai/knowledgebase/articles/781422-how-to-import-data-for-use-by-apache-zeppelin but drag&drop does not work as expected. Best regards, Uwe -- Best Regards Jeff Zhang
Can't get Cassandra and Spark 3.0.0 to work
Hi
I am trying to get
* Apache Spark 3.0
* Spark Cassandra Connector 3.0.0
To work together from Zeppelin, but are running into issues that seems to be
related to Scala 2.12
Like for example NoSuchMethodError, see below
Has anyone been able to get this to work ?
There are files in the Zeppelin project that kind of indicates that this should
indeed work, but I can’t figure out what I am doing wrong here
What I have done is
1. Using Zeppelin Docker
2. Installed Spark 3.0 inside of Docker
3. Set SPARK_HOME=/opt/spark (pointing to the newly installed Spark 3)
4. In Zeppelin I have added the dependency for Spark
com.datastax.spark:spark-cassandra-connector_2.12:3.0.0
I can see that inside of the directory :
/opt/zeppelin/local-repo/spark
There are a mix of 2.11 and 2.12 libraries
scala-library-2.11.12.jar
scala-logging_2.11-3.5.0.jar
scala-reflect-2.11.12.jar
I suspect that there is a Scala 2.11 vs 2.12 issue here, but I am at the moment
unable to figure out how to fix it
So any help would be much appreciated 😊
-Tobias
zeppelin | INFO [2021-07-01 08:34:56,064] ({SchedulerFactory60}
RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:393) - Push local
angular object registry from ZeppelinServer to remote interpreter group
spark-shared_process
zeppelin | INFO [2021-07-01 08:34:56,146]
({JobStatusPoller-paragraph_1625079989375_1248319040}
NotebookServer.java[onStatusChange]:1931) - Job
paragraph_1625079989375_1248319040 starts to RUNNING
zeppelin | INFO [2021-07-01 08:34:56,147]
({JobStatusPoller-paragraph_1625079989375_1248319040}
VFSNotebookRepo.java[save]:144) - Saving note 2GAM4MGRX to T3_A_2GAM4MGRX.zpln
zeppelin | WARN [2021-07-01 08:35:03,829] ({SchedulerFactory60}
NotebookServer.java[onStatusChange]:1928) - Job
paragraph_1625079989375_1248319040 is finished, status: ERROR, exception: null,
result: %text org.apache.zeppelin.interpreter.InterpreterException:
java.lang.BootstrapMethodError: java.lang.NoSuchMethodError:
scala.collection.TraversableOnce.$anonfun$count$1$adapted(Lscala/Function1;Lscala/runtime/IntRef;Ljava/lang/Object;)Ljava/lang/Object;
zeppelin | at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:76)
zeppelin | at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:836)
zeppelin | at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:744)
zeppelin | at org.apache.zeppelin.scheduler.Job.run(Job.java:172)
zeppelin | at
org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:132)
zeppelin | at
org.apache.zeppelin.scheduler.FIFOScheduler.lambda$runJobInScheduler$0(FIFOScheduler.java:42)
zeppelin | at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
zeppelin | at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
zeppelin | at java.lang.Thread.run(Thread.java:748)
zeppelin | Caused by: java.lang.BootstrapMethodError:
java.lang.NoSuchMethodError:
scala.collection.TraversableOnce.$anonfun$count$1$adapted(Lscala/Function1;Lscala/runtime/IntRef;Ljava/lang/Object;)Ljava/lang/Object;
zeppelin | at
scala.tools.nsc.interpreter.Naming.unmangle(Naming.scala:30)
zeppelin | at
scala.tools.nsc.interpreter.Naming.unmangle$(Naming.scala:24)
zeppelin | at
scala.tools.nsc.interpreter.IMain$naming$.unmangle(IMain.scala:188)
zeppelin | at
scala.tools.nsc.interpreter.IMain$ReplStrippingWriter.stripImpl(IMain.scala:1340)
zeppelin | at
scala.tools.nsc.interpreter.IMain$StrippingWriter.strip(IMain.scala:1314)
zeppelin | at
scala.tools.nsc.interpreter.IMain$StrippingWriter.strip$(IMain.scala:1314)
zeppelin | at
scala.tools.nsc.interpreter.IMain$StrippingTruncatingWriter.strip(IMain.scala:1325)
zeppelin | at
scala.tools.nsc.interpreter.IMain$StrippingTruncatingWriter.clean(IMain.scala:1331)
zeppelin | at
scala.tools.nsc.interpreter.IMain$StrippingTruncatingWriter.write(IMain.scala:1332)
zeppelin | at java.io.PrintWriter.print(PrintWriter.java:603)
zeppelin | at java.io.PrintWriter.println(PrintWriter.java:739)
zeppelin | at
scala.tools.nsc.reporters.ConsoleReporter.printMessage(ConsoleReporter.scala:49)
zeppelin | at
scala.tools.nsc.interpreter.ReplReporter.printMessage(ReplReporter.scala:93)
zeppelin | at
scala.tools.nsc.interpreter.IMain.$anonfun$interpret$1(IMain.scala:582)
zeppelin | at
scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:41)
zeppelin | at
scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:37)
zeppelin | at
scala.reflect.internal.uti
Re: Can't get Cassandra and Spark 3.0.0 to work
/opt/zeppelin/local-repo/spark is the location where zeppelin download
dependencies, if you find both scala-2.11 and scala-2.12, then I suspect
besides com.datastax.spark:spark-cassandra-connector_2.12:3.0.0 what else
dependencies do you use ?
Tobias Eriksson 于2021年7月1日周四 下午10:51写道：
> Hi
>
> I am trying to get
>
>- Apache Spark 3.0
>- Spark Cassandra Connector 3.0.0
>
> To work together from Zeppelin, but are running into issues that seems to
> be related to Scala 2.12
>
> Like for example NoSuchMethodError, see below
>
>
>
> Has anyone been able to get this to work ?
>
> There are files in the Zeppelin project that kind of indicates that this
> should indeed work, but I can’t figure out what I am doing wrong here
>
>
>
> What I have done is
>
>1. Using Zeppelin Docker
>2. Installed Spark 3.0 inside of Docker
>3. Set SPARK_HOME=/opt/spark (pointing to the newly installed Spark 3)
>4. In Zeppelin I have added the dependency for Spark
>com.datastax.spark:spark-cassandra-connector_2.12:3.0.0
>
>
>
> I can see that inside of the directory :
>
>
>
> /opt/zeppelin/local-repo/spark
>
>
>
> There are a mix of 2.11 and 2.12 libraries
>
>
>
> scala-library-2.11.12.jar
>
> scala-logging_2.11-3.5.0.jar
>
> scala-reflect-2.11.12.jar
>
>
>
> I suspect that there is a Scala 2.11 vs 2.12 issue here, but I am at the
> moment unable to figure out how to fix it
>
> So any help would be much appreciated 😊
>
>
>
> -Tobias
>
>
>
> *zeppelin* *|* INFO [2021-07-01 08:34:56,064]
> ({SchedulerFactory60}
> RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:393) - Push local
> angular object registry from ZeppelinServer to remote interpreter group
> spark-shared_process
>
> *zeppelin* *|* INFO [2021-07-01 08:34:56,146]
> ({JobStatusPoller-paragraph_1625079989375_1248319040}
> NotebookServer.java[onStatusChange]:1931) - Job
> paragraph_1625079989375_1248319040 starts to RUNNING
>
> *zeppelin* *|* INFO [2021-07-01 08:34:56,147]
> ({JobStatusPoller-paragraph_1625079989375_1248319040}
> VFSNotebookRepo.java[save]:144) - Saving note 2GAM4MGRX to
> T3_A_2GAM4MGRX.zpln
>
> *zeppelin* *|* WARN [2021-07-01 08:35:03,829]
> ({SchedulerFactory60} NotebookServer.java[onStatusChange]:1928) - Job
> paragraph_1625079989375_1248319040 is finished, status: ERROR, exception:
> null, result: %text org.apache.zeppelin.interpreter.InterpreterException:
> java.lang.BootstrapMethodError: java.lang.NoSuchMethodError:
> scala.collection.TraversableOnce.$anonfun$count$1$adapted(Lscala/Function1;Lscala/runtime/IntRef;Ljava/lang/Object;)Ljava/lang/Object;
>
> *zeppelin* *|* at
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:76)
>
> *zeppelin* *|* at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:836)
>
> *zeppelin* *|* at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:744)
>
> *zeppelin* *|* at
> org.apache.zeppelin.scheduler.Job.run(Job.java:172)
>
> *zeppelin* *|* at
> org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:132)
>
> *zeppelin* *|* at
> org.apache.zeppelin.scheduler.FIFOScheduler.lambda$runJobInScheduler$0(FIFOScheduler.java:42)
>
> *zeppelin* *|* at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>
> *zeppelin* *|* at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>
> *zeppelin* *|* at java.lang.Thread.run(Thread.java:748)
>
> *zeppelin* *|* Caused by: java.lang.BootstrapMethodError:
> java.lang.NoSuchMethodError:
> scala.collection.TraversableOnce.$anonfun$count$1$adapted(Lscala/Function1;Lscala/runtime/IntRef;Ljava/lang/Object;)Ljava/lang/Object;
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.Naming.unmangle(Naming.scala:30)
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.Naming.unmangle$(Naming.scala:24)
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.IMain$naming$.unmangle(IMain.scala:188)
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.IMain$ReplStrippingWriter.stripImpl(IMain.scala:1340)
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.IMain$StrippingWriter.strip(IMain.scala:1314)
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.IMain$StrippingWriter.strip$(IMain.scala:1314)
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.IMain$StrippingTruncatingWriter.strip(IMain.scala:1325)
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.IMain$StrippingTruncatingWriter.clean(IMain.scala:1331)
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.IMain$StrippingTruncatingWriter.write(IMain.scala:1332)
>
> *zeppelin* *|* at java.io.PrintWriter.print(PrintWriter.java:603)
>
> *zeppelin* *|* at
> java.io.Pri
Re: Can't get Cassandra and Spark 3.0.0 to work
try to use assembly version of SCC, and 3.0.1:
https://mvnrepository.com/artifact/com.datastax.spark/spark-cassandra-connector-assembly_2.12/3.0.1
- it definitely works with Spark 3.0, not clear from where the Scala 2.11
is coming
On Thu, Jul 1, 2021 at 4:50 PM Tobias Eriksson
wrote:
> Hi
>
> I am trying to get
>
>- Apache Spark 3.0
>- Spark Cassandra Connector 3.0.0
>
> To work together from Zeppelin, but are running into issues that seems to
> be related to Scala 2.12
>
> Like for example NoSuchMethodError, see below
>
>
>
> Has anyone been able to get this to work ?
>
> There are files in the Zeppelin project that kind of indicates that this
> should indeed work, but I can’t figure out what I am doing wrong here
>
>
>
> What I have done is
>
>1. Using Zeppelin Docker
>2. Installed Spark 3.0 inside of Docker
>3. Set SPARK_HOME=/opt/spark (pointing to the newly installed Spark 3)
>4. In Zeppelin I have added the dependency for Spark
>com.datastax.spark:spark-cassandra-connector_2.12:3.0.0
>
>
>
> I can see that inside of the directory :
>
>
>
> /opt/zeppelin/local-repo/spark
>
>
>
> There are a mix of 2.11 and 2.12 libraries
>
>
>
> scala-library-2.11.12.jar
>
> scala-logging_2.11-3.5.0.jar
>
> scala-reflect-2.11.12.jar
>
>
>
> I suspect that there is a Scala 2.11 vs 2.12 issue here, but I am at the
> moment unable to figure out how to fix it
>
> So any help would be much appreciated 😊
>
>
>
> -Tobias
>
>
>
> *zeppelin* *|* INFO [2021-07-01 08:34:56,064]
> ({SchedulerFactory60}
> RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:393) - Push local
> angular object registry from ZeppelinServer to remote interpreter group
> spark-shared_process
>
> *zeppelin* *|* INFO [2021-07-01 08:34:56,146]
> ({JobStatusPoller-paragraph_1625079989375_1248319040}
> NotebookServer.java[onStatusChange]:1931) - Job
> paragraph_1625079989375_1248319040 starts to RUNNING
>
> *zeppelin* *|* INFO [2021-07-01 08:34:56,147]
> ({JobStatusPoller-paragraph_1625079989375_1248319040}
> VFSNotebookRepo.java[save]:144) - Saving note 2GAM4MGRX to
> T3_A_2GAM4MGRX.zpln
>
> *zeppelin* *|* WARN [2021-07-01 08:35:03,829]
> ({SchedulerFactory60} NotebookServer.java[onStatusChange]:1928) - Job
> paragraph_1625079989375_1248319040 is finished, status: ERROR, exception:
> null, result: %text org.apache.zeppelin.interpreter.InterpreterException:
> java.lang.BootstrapMethodError: java.lang.NoSuchMethodError:
> scala.collection.TraversableOnce.$anonfun$count$1$adapted(Lscala/Function1;Lscala/runtime/IntRef;Ljava/lang/Object;)Ljava/lang/Object;
>
> *zeppelin* *|* at
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:76)
>
> *zeppelin* *|* at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:836)
>
> *zeppelin* *|* at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:744)
>
> *zeppelin* *|* at
> org.apache.zeppelin.scheduler.Job.run(Job.java:172)
>
> *zeppelin* *|* at
> org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:132)
>
> *zeppelin* *|* at
> org.apache.zeppelin.scheduler.FIFOScheduler.lambda$runJobInScheduler$0(FIFOScheduler.java:42)
>
> *zeppelin* *|* at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>
> *zeppelin* *|* at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>
> *zeppelin* *|* at java.lang.Thread.run(Thread.java:748)
>
> *zeppelin* *|* Caused by: java.lang.BootstrapMethodError:
> java.lang.NoSuchMethodError:
> scala.collection.TraversableOnce.$anonfun$count$1$adapted(Lscala/Function1;Lscala/runtime/IntRef;Ljava/lang/Object;)Ljava/lang/Object;
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.Naming.unmangle(Naming.scala:30)
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.Naming.unmangle$(Naming.scala:24)
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.IMain$naming$.unmangle(IMain.scala:188)
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.IMain$ReplStrippingWriter.stripImpl(IMain.scala:1340)
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.IMain$StrippingWriter.strip(IMain.scala:1314)
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.IMain$StrippingWriter.strip$(IMain.scala:1314)
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.IMain$StrippingTruncatingWriter.strip(IMain.scala:1325)
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.IMain$StrippingTruncatingWriter.clean(IMain.scala:1331)
>
> *zeppelin* *|* at
> scala.tools.nsc.interpreter.IMain$StrippingTruncatingWriter.write(IMain.scala:1332)
>
> *zeppelin* *|* at java.io.PrintWriter.print(PrintWriter.java:603)
>
> *zeppelin* *|* at
> java.io.Pr
How to disable creating a new notebook for few users
I am using Zeppelin 0.8.2, different kinds of users use our zeppelin. There is one category of users wherein they just need to run the notebook. I know how to set the authorization to disable 'edit or running' the notebook but I did not find the option to disable creating a new notebook for these users. Thanks, Cod
Re: How to disable creating a new notebook for few users
I don't think zeppelin has such a feature yet. Great Info 于2021年7月2日周五 上午2:42写道： > I am using Zeppelin 0.8.2, different kinds of users use our zeppelin. > There is one category of users wherein they just need to run the notebook. > I know how to set the authorization to disable 'edit or running' the > notebook but I did not find the option to disable creating a new notebook > for these users. > > Thanks, > Cod > -- Best Regards Jeff Zhang
