David Lacalle Castillo created ZEPPELIN-4919:
------------------------------------------------
Summary: My spark executors have GC storage connector but is not
loaded
Key: ZEPPELIN-4919
URL: https://issues.apache.org/jira/browse/ZEPPELIN-4919
Project: Zeppelin
Issue Type: Bug
Components: spark
Affects Versions: 0.9.0
Reporter: David Lacalle Castillo
I have the following code:
%spark.dep
z.addRepo("maven").url("https://mvnrepository.com/";)
z.load("com.google.cloud.bigdataoss:gcs-connector:jar:hadoop3-2.1.3")
However, when I try to load data from google storage it trhows me next error:
```
Py4JJavaError: An error occurred while calling o72.csv. :
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem not found at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195) at
org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2654) at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667) at
org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94) at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703) at
org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685) at
org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373) at
org.apache.hadoop.fs.Path.getFileSystem(Path.java:295) at
org.apache.spark.sql.execution.datasources.DataSource$$anonfun$org$apache$spark$sql$execution$datasources$DataSource$$checkAndGlobPathIfNecessary$1.apply(DataSource.scala:547)
at
org.apache.spark.sql.execution.datasources.DataSource$$anonfun$org$apache$spark$sql$execution$datasources$DataSource$$checkAndGlobPathIfNecessary$1.apply(DataSource.scala:545)
at
scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at
scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.immutable.List.foreach(List.scala:392) at
scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241) at
scala.collection.immutable.List.flatMap(List.scala:355) at
org.apache.spark.sql.execution.datasources.DataSource.org$apache$spark$sql$execution$datasources$DataSource$$checkAndGlobPathIfNecessary(DataSource.scala:545)
at
org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:359)
at
org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223) at
org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211) at
org.apache.spark.sql.DataFrameReader.csv(DataFrameReader.scala:618) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at
py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at
py4j.Gateway.invoke(Gateway.java:282) at
py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at
py4j.commands.CallCommand.execute(CallCommand.java:79) at
py4j.GatewayConnection.run(GatewayConnection.java:238) at
java.lang.Thread.run(Thread.java:748) Caused by:
java.lang.ClassNotFoundException: Class
com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem not found at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101) at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193) ... 30
more (<class 'py4j.protocol.Py4JJavaError'>, Py4JJavaError('An error occurred
while calling o72.csv.\n', JavaObject id=o75), <traceback object at
0x7f6e4df09b88>)
```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
