Hi Deepak, When I use Spark with Hive, this is the setup I follow. It should work with whichever version of Spark you choose.
cp /etc/hive/conf/hive-site.xml spark-1.3.1-bin-hadoop2.6/conf/ # Disable ATS hooks & Tez sed -i "s/org\.apache\.hadoop\.hive\.ql\.hooks\.ATSHook//g" spark-1.3.1-bin-hadoop2.6/conf/hive-site.xml sed -i "s/<value>tez/<value>mr/g" spark-1.3.1-bin-hadoop2.6/conf/hive-site.xml sed -ri 's ([0-9]+)s \1 g' spark-1.3.1-bin-hadoop2.6/conf/hive-site.xml Thanks, -Randy From: "ÐΞ€ρ@Ҝ (๏̯͡๏)" Reply-To: "[email protected]<mailto:[email protected]>" Date: Tuesday, August 4, 2015 at 2:37 PM To: "[email protected]<mailto:[email protected]>" Subject: Re: Zeppelin Tutorial Notebook Fails Fixed hive-site.xml from zeppelin conf <property> <name>hive.metastore.client.connect.retry.delay</name> <value>5</value> </property> <property> <name>hive.metastore.client.socket.timeout</name> <value>1800</value> </property> Remove s (postfix). Now i see this exception java.lang.NoClassDefFoundError: org/apache/tez/dag/api/SessionNotRunning at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:353) at org.apache.spark.sql.hive.HiveContext.sessionState$lzycompute(HiveContext.scala:239) at org.apache.spark.sql.hive.HiveContext.sessionState(HiveContext.scala:235) at org.apache.spark.sql.hive.HiveContext$QueryExecution.<init>(HiveContext.scala:383) at org.apache.spark.sql.hive.HiveContext.executePlan(HiveContext.scala:89) at org.apache.spark.sql.hive.HiveContext.executePlan(HiveContext.scala:49) at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:131) at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:51) at org.apache.spark.sql.SQLContext.createDataFrame(SQLContext.scala:323) Questions 1) Load Data table does not specify any interpreter at the beginning. What is the default one ? (%spark ?) 2) How does hive and later Tez come into picture here as am submitting a spark job ? Regards, Deepak On Tue, Aug 4, 2015 at 11:29 AM, ÐΞ€ρ@Ҝ (๏̯͡๏) <[email protected]<mailto:[email protected]>> wrote: Load Data Into Table works fine. But But the next paragraph %sql select age, count(1) value from bank where age < 30 group by age order by age Fails java.lang.NumberFormatException: For input string: "5s" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:492) at java.lang.Integer.parseInt(Integer.java:527) at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1258) Even when i change the query to select age from bank it throws same exception. Any suggestions ? -- Deepak -- Deepak
