Greetings Attached is the results of the select count(1) from table The contents of the dataset(table) is 18 columns and 3340 rows
hive> select count(1) from commoncrawl18; Query ID = jmill383_20160520074710_3b5ee662-2ead-4d89-9123-df9b2cf6e2d7 Total jobs = 1 Launching Job 1 out of 1 Number of reduce tasks determined at compile time: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer=<number> In order to limit the maximum number of reducers: set hive.exec.reducers.max=<number> In order to set a constant number of reducers: set mapreduce.job.reduces=<number> Starting Job = job_1463594979064_0005, Tracking URL = http://starchild:8088/proxy/application_1463594979064_0005/ Kill Command = /opt/hadoop/bin/hadoop job -kill job_1463594979064_0005 Hadoop job information for Stage-1: number of mappers: 0; number of reducers: 0 2016-05-20 07:47:15,936 Stage-1 map = 0%, reduce = 0% Ended Job = job_1463594979064_0005 with errors Error during job, obtaining debugging information... Job Tracking URL: http://starchild:8088/cluster/app/application_1463594979064_0005 FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask MapReduce Jobs Launched: Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 FAIL Total MapReduce CPU Time Spent: 0 msec hive> On Thu, May 19, 2016 at 8:56 AM, Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > Hi John, > > stderr does not say much > > Exception in thread "main" java.lang.IncompatibleClassChangeError: > Implementing class > > at java.lang.ClassLoader.defineClass1(Native Method) > > at java.lang.ClassLoader.defineClass(ClassLoader.java:800) > > at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > > at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) > > at java.net.URLClassLoader.access$100(URLClassLoader.java:71) > > at java.net.URLClassLoader$1.run(URLClassLoader.java:361) > > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > > at java.security.AccessController.doPrivileged(Native Method) > > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > > at java.lang.Class.getDeclaredMethods0(Native Method) > > at java.lang.Class.privateGetDeclaredMethods(Class.java:2615) > > at java.lang.Class.getMethod0(Class.java:2856) > > at java.lang.Class.getMethod(Class.java:1668) > > at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494) > > at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486) > > > However, sounds like you may have an issue with yarn container memory. > > > How big is the underlying table. Also can you just do a plain select > count(1) from <table> itself (no distinct etc) and see it works? > > > HTH > > > > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.wordpress.com > > > > On 18 May 2016 at 19:46, JOHN MILLER <jmill...@gmail.com> wrote: > >> Mich >> >> Attaching hadoop logs >> >> John M >> >> On Wed, May 18, 2016 at 1:48 PM, Mich Talebzadeh < >> mich.talebza...@gmail.com> wrote: >> >>> Hi John, >>> >>> can you please a new thread for your problem so we can deal with >>> separately. >>> >>> thanks >>> >>> Dr Mich Talebzadeh >>> >>> >>> >>> LinkedIn * >>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >>> >>> >>> >>> http://talebzadehmich.wordpress.com >>> >>> >>> >>> On 18 May 2016 at 15:11, JOHN MILLER <jmill...@gmail.com> wrote: >>> >>>> Greetings Mitch >>>> >>>> I have an issue with running mapreduce in hive I am getting a >>>> FAILED: Execution Error, return code 2 from >>>> org.apache.hadoop.hive.ql.exec.mr.MapRedTask >>>> >>>> error while attemtiing to execute SELECT DISTINCT(fieldname) FROM TABLE >>>> x or SELECT COUNT(*) FROM TABLE x;; Trying to run cascading-hive gives >>>> me the same problem as well >>>> >>>> Please advise if u have come across this type of problem or generated >>>> some ideas as to resolve this problema >>>> >>>> On Wed, May 18, 2016 at 9:53 AM, Mich Talebzadeh < >>>> mich.talebza...@gmail.com> wrote: >>>> >>>>> Hi Kuldeep, >>>>> >>>>> Have you installed hive on any of these nodes. >>>>> >>>>> Hive is basically an API. You will also need to install sqoop as well >>>>> if you are going to import data from other RDBMss like Oracle, Sybase etc. >>>>> >>>>> Hive has a very small footprint so my suggestion is to install it on >>>>> all your boxes and permission granted to Haddop user say hduser. >>>>> >>>>> Hive will require a metadata in a database of your choice. default is >>>>> derby which I don't use. try to use a reasonable database. ours is on >>>>> Oracle >>>>> >>>>> Now under directory $HIVE_HOME/conf/hive-site.xml you can set up info >>>>> about Hadoop and your metastore etc. You also need to set up environment >>>>> variables for both Hadoop and hive in your start up script like .profile >>>>> .kshrc etc >>>>> >>>>> Have a look anyway. >>>>> >>>>> HTH >>>>> >>>>> Dr Mich Talebzadeh >>>>> >>>>> >>>>> >>>>> LinkedIn * >>>>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >>>>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >>>>> >>>>> >>>>> >>>>> http://talebzadehmich.wordpress.com >>>>> >>>>> >>>>> >>>>> On 18 May 2016 at 13:49, Kuldeep Chitrakar < >>>>> kuldeep.chitra...@synechron.com> wrote: >>>>> >>>>>> I have a very basic question regarding Hadoop & Hive setup. I have 7 >>>>>> Machines say M1,M2,M3,M4,M5,M6,M7 >>>>>> >>>>>> >>>>>> >>>>>> Hadoop Cluster Setup: >>>>>> >>>>>> >>>>>> >>>>>> Namenode: M1 >>>>>> >>>>>> Seondary Namenode: M2 >>>>>> >>>>>> Datanodes: M3,M4,M5 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Now question is: >>>>>> >>>>>> >>>>>> >>>>>> Where do I need to install Hive. >>>>>> >>>>>> 1. Should I install Hiverserver on M6 >>>>>> >>>>>> a. if yes does that machine needs core Hadoop JAR’s installed? >>>>>> >>>>>> b. How this Hive server knows where Hadoop cluster is. What >>>>>> configurations needs to be done? >>>>>> >>>>>> c. How can we restrict this machine to be only hive server and >>>>>> not datanode of Hadoop cluster? >>>>>> >>>>>> >>>>>> >>>>>> 2. Where do we install Hive CLI >>>>>> >>>>>> a. If I want to hive M7 as Hive CLI, then what needs to be >>>>>> installed on this machine. >>>>>> >>>>>> b. Any required configurations. >>>>>> >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Kuldeep >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>> >> >
