Thanks, that did seem to make a difference. I am a bit scared of this approach as spark itself has a different guava dependency but the error does go away this way
On Wed, Jun 24, 2015 at 10:04 AM, Akhil Das <[email protected]> wrote: > Can you try to add those jars in the SPARK_CLASSPATH and give it a try? > > Thanks > Best Regards > > On Wed, Jun 24, 2015 at 12:07 AM, Yana Kadiyska <[email protected]> > wrote: > >> Hi folks, I have been using Spark against an external Metastore service >> which runs Hive with Cdh 4.6 >> >> In Spark 1.2, I was able to successfully connect by building with the >> following: >> >> ./make-distribution.sh --tgz -Dhadoop.version=2.0.0-mr1-cdh4.2.0 >> -Phive-thriftserver -Phive-0.12.0 >> >> I see that in Spark 1.4 the Hive 0.12.0 profile is deprecated in favor of >> spark.sql.hive.metastore.version/spark.sql.hive.metastore.jars >> >> When I tried to use this setup spark-shell fails for me with the >> following error: >> >> 15/06/23 18:18:07 INFO hive.HiveContext: Initializing >> HiveMetastoreConnection version 0.12.0 using [Ljava.net.URL;@7b7a9a6c >> java.lang.ClassNotFoundException: java.lang.NoClassDefFoundError: >> com/google/common/base/Preconditions when creating Hive client using >> classpath: file:/hive/lib/guava-11.0.2.jar, >> file:/hive/lib/hive-exec-0.10.0-cdh4.6.0.jar, >> file:/hive/lib/hive-metastore-0.10.0-cdh4.6.0.jar, >> file:/hadoop/share/hadoop/mapreduce1/lib/hadoop-common-2.0.0-cdh4.6.0.jar, >> file:/hive/lib/commons-logging-1.0.4.jar >> >> >> >> I don't know why it's not seeing the class -- it's in the guava jar. If >> anyone has had success with 0.12 version please let me know what jars need >> to be on the classpath. I think my Hive version might be too outdated but I >> don't control the metastore and I had success with Spark1.2 so I'm hoping... >> >> >> >
