Julien Buret created ZEPPELIN-297:
-------------------------------------
Summary: Dependency should be loaded in pypsark
Key: ZEPPELIN-297
URL: https://issues.apache.org/jira/browse/ZEPPELIN-297
Project: Zeppelin
Issue Type: Bug
Components: Interpreters
Affects Versions: 0.6.0
Reporter: Julien Buret
dependency loaded with %dep should be added in pyspark
Exemple:
//Dataframe csv reader added as dependency
%dep
z.reset()
z.load("com.databricks:spark-csv_2.11:1.2.0")
// Csv reader can be used in scala
import org.apache.spark.sql.SQLContext
val sqlContext = new SQLContext(sc)
val df = sqlContext.read.format("com.databricks.spark.csv").option("header",
"true").load("train.csv")
z.show(df)
// But not with pyspark
%pyspark
from pyspark.sql import SQLContext
sqlsc = SQLContext(sc)
sqlsc.read.format('com.databricks.spark.csv').load('train.csv')
Py4JJavaError: An error occurred while calling o57.load.
: java.lang.RuntimeException: Failed to load class for data source:
com.databricks.spark.csv
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
