Someone has any idea about this subject?
> On 20 Aug 2015, at 00:47, Yusuf Can Gürkan wrote:
>
> Hey,
>
> This is my spark-env:
>
> # Add Hadoop libraries to Spark classpath
> SPARK_CLASSPATH="${SPARK_CLASSPATH}:${HADOOP_HOME}/*:${HADOOP_HOME}/../hadoop-hdfs/*:${HADOOP_HOME}/../hadoop-mapreduc
Hey,
This is my spark-env:
# Add Hadoop libraries to Spark classpath
SPARK_CLASSPATH="${SPARK_CLASSPATH}:${HADOOP_HOME}/*:${HADOOP_HOME}/../hadoop-hdfs/*:${HADOOP_HOME}/../hadoop-mapreduce/*:${HADOOP_HOME}/../hadoop-yarn/*"
LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${HADOOP_HOME}/lib/native"
# Add Hiv
Hey Yin,
Thanks for answer. I thought that this could be problem but i can not create
HiveContext because i can not import org.apache.spark.sql.hive.HiveContext. It
does not see this package.
I read that i should build spark with -PHive but i’m running on Amazon EMR
1.4.1 and on spark-shell i
Can you try to use HiveContext instead of SQLContext? Your query is trying
to create a table and persist the metadata of the table in metastore, which
is only supported by HiveContext.
On Wed, Aug 19, 2015 at 8:44 AM, Yusuf Can Gürkan
wrote:
> Hello,
>
> I’m trying to create a table with sqlCont
Hello,
I’m trying to create a table with sqlContext.sql method as below:
val sc = new SparkContext()
val sqlContext = new SQLContext(sc)
import sqlContext.implicits._
sqlContext.sql(s"""
create table if not exists landing (
date string,
referrer string
)
partitioned by (partnerid string,dt stri
