Re: Hive on Spark - Hadoop 2 - Installation - Ubuntu

2015-11-26 Thread Dasun Hegoda
This works fine for me spark-shell --master yarn-client On Tue, Nov 24, 2015 at 11:43 AM, Dasun Hegoda wrote: > Hey floks, > > Any updates? > > On Mon, Nov 23, 2015 at 5:15 PM, Dasun Hegoda > wrote: > >> Do you have any clue how to get his fixed? >> >> On Mon, Nov 23, 2015 at 4:27 PM, Dasun He

Re: ERROR util.SparkUncaughtExceptionHandler: Uncaught exception in thread Thread

2015-11-26 Thread Dasun Hegoda
Any updates guys? On Thu, Nov 26, 2015 at 10:20 AM, Dasun Hegoda wrote: > Anything? > > On Tue, Nov 24, 2015 at 11:49 PM, Dasun Hegoda > wrote: > >> I'm using *apache-hive-1.2.1-bin *with *spark-1.5.1-bin-hadoop2.6 *on >> top of >> *hadoop-2.7.1 *. Any idea how to fix this error? >> >> On Tue,

RE: Hive version with Spark

2015-11-26 Thread Mich Talebzadeh
Hi Sophia, There is no Hadoop-2.6. I believe you should use Hadoop-2.4 as shown below mvn -Phadoop-2.4 -Dhadoop.version=2.6.0 -DskipTests clean package Also if you are building it for Hive on Spark engine, you should not include Hadoop.jar files in your build. For example I tr

Rule Transformation in Hadoop

2015-11-26 Thread Mahender SNPC
Hi team, We need expert input to discuss how to implement Rule engine in hive. Do you have any references available to implement rule in hive/pig. We are migrating our Stored Procedures into Multiple Hive query but it is becoming complex in maintenance, Hive is not Procedural Language, so w

RE: Need you Expert inputs

2015-11-26 Thread Mich Talebzadeh
Hi Mahender How have you create your ORC table? Use show create table and send the output Mich Talebzadeh Sybase ASE 15 Gold Medal Award 2008 A Winning Strategy: Running the most Critical Financial Data on ASE 15

RE: Need you Expert inputs

2015-11-26 Thread Mahender SNPC
Hi Team, We need your expert input in resolving issue while updating record in Hive ORC bucket table. I’m getting “FAILED: SemanticException [Error 10294]: Attempt to do update or delete using transaction manager that does not support these operations” . I'm in HDP 2.3 Here are the ste

building spark from 1.3 release without Hive

2015-11-26 Thread Mich Talebzadeh
Hi, I am not having much luck making Hive run on Spark! I tried to build spark 1.5.2 without Hive jards. It worked but could not run hive sql on Spark. I saw in this link: http://stackoverflow.com/questions/33233431/hive-on-spark-java-lang-noclassdeffounderror-org-apache-hive-spark-cli

Java heap space occured when the amount of data is very large with the same key on join sql

2015-11-26 Thread Jone Zhang
Here is an error message: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2245) at java.util.Arrays.copyOf(Arrays.java:2219) at java.util.ArrayList.grow(ArrayList.java:242) at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:216) at java.util.ArrayList.e