spark access old version of Hadoop 2.1.0 and Hive version 0.11

2016-01-09 Thread Jade Liu
Hi, All: I'm trying to read and write from the hdfs cluster using SparkSQL hive context. My current build of spark is 1.5.2. The problem is that currently our company has very old version of hdfs (hadoop 2.1.0) and hive metastore (0.11) using Hortonworks bundle. One of the possible solution i

Re: problem building spark on centos

2016-01-06 Thread Jade Liu
Yes I’m using maven 3.3.9. From: Todd Nist mailto:tsind...@gmail.com>> Date: Wednesday, January 6, 2016 at 12:33 PM To: Jade Liu mailto:jade@nor1.com>> Cc: "user@spark.apache.org<mailto:user@spark.apache.org>" mailto:user@spark.apache.org>> Subject: Re: pro

Re: problem building spark on centos

2016-01-06 Thread Jade Liu
ava problem? I’m using oracle JDK 1.7. Should I update it to 1.8 instead? I just checked the conf and it says 1.7. Thanks, Jade From: Todd Nist mailto:tsind...@gmail.com>> Date: Wednesday, January 6, 2016 at 12:03 PM To: Jade Liu mailto:jade@nor1.com>> Cc: Ted Yu mailto:yuzhih...@gmai

Re: problem building spark on centos

2016-01-06 Thread Jade Liu
nks! Jade From: Ted Yu mailto:yuzhih...@gmail.com>> Date: Wednesday, January 6, 2016 at 10:40 AM To: Jade Liu mailto:jade@nor1.com>>, user mailto:user@spark.apache.org>> Subject: Re: problem building spark on centos w.r.t. the second error, have you read this ? http://www.cap

problem building spark on centos

2016-01-05 Thread Jade Liu
Hi, All: I'm trying to build spark 1.5.2 from source using maven with the following command: ./make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0 -Dscala-2.11 -Phive -Phive-thriftserver -DskipTests I got the following error: + VERSION='[ERROR] [Help 2] http://cwiki.apache.or