Hitesh: I have attached application logs.
On Tue, May 13, 2014 at 8:43 PM, Carter Shanklin <car...@hortonworks.com>wrote: > Archit, > > I have seen this problem on small sandbox environments with very little > memory given to YARN. In those cases, the app master would consume enough > memory that there was no memory left to launch containers. Can you confirm > that you have enough memory to launch a few containers? > > > On Sun, May 11, 2014 at 9:59 PM, Archit Thakur > <archit279tha...@gmail.com>wrote: > >> Hi Bikas, >> >> I was able to resolve this issue, by copying hive-exec.jar to >> hdfs://user/admin/, It was not able to localize it while running Tez. >> >> Though, I am facing some other problem now. My Tez job is not progressing, >> >> My Tez job doesn't progress, it just keeps printing: >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> Map 1: 0/2 Map 5: 0/2 Reducer 2: 0/1 Reducer 3: 0/1 Reducer >> 4: 0/1 >> >> In hive logs also, it keeps on printing the same. >> >> Any idea, where should I look into or how should I debug the problem. Or >> pointers on problem itself would help. >> Thanks Archit. >> >> >> On Sun, May 11, 2014 at 7:28 AM, Bikas Saha <bi...@hortonworks.com>wrote: >> >>> Setting execution engine to mr and framework name to yarn = Hive >>> compiles to MR and runs on MR. >>> >>> Setting execution engine to mr and framework name to yarn-tez = Hive >>> compiles to MR and runs on Tez. >>> >>> Setting execution engine to tez = Hive compiles to Tez and runs on Tez. >>> >>> >>> >>> Did you change the execution engine in the same shell that was earlier >>> running with execution engine set to mr? If yes, then try and set the >>> execution engine in hive-site.xml and use a new hive shell. >>> >>> Have you set all required hive configurations? >>> >>> >>> >>> If you can still reproduce the issue, then you should create a jira in >>> the Apache hive project to track the bug. >>> >>> >>> >>> Bikas >>> >>> >>> >>> *From:* Archit Thakur [mailto:archit279tha...@gmail.com] >>> *Sent:* Thursday, May 08, 2014 4:42 AM >>> *To:* user@hive.apache.org; u...@tez.incubator.apache.org >>> *Subject:* Tez HIve problems. >>> >>> >>> >>> Hi, >>> >>> I am facing few problems while using Tez on Hive. >>> >>> I am able to run Tez through YARN submitting independent MR procesing >>> job. >>> >>> Also, when I run SELECT query on hive shell by setting >>> >>> *set hive.execution.engine=mr;* >>> >>> but setting *mapreduce.framework.name >>> <http://mapreduce.framework.name>=yarn-tez* >>> >>> This launches tez job, I checked by seeing AM's UI of machine. >>> >>> But when I *set hive.execution.engine=tez;* >>> >>> >>> >>> *It gives me *Query ID = >>> admin_20140508112525_37cf7b55-75ac-408c-b96f-8e40ba2f8e2b >>> Total jobs = 1 >>> Launching Job 1 out of 1 >>> FAILED: Execution Error, return code 1 from >>> org.apache.hadoop.hive.ql.exec.tez.TezTask >>> >>> *I checked the hive logs:* >>> >>> exec.Task (TezTask.java:execute(185)) - Failed to execute tez graph. >>> java.io.FileNotFoundException: File does not exist: hdfs:/user/admin >>> at >>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1110) >>> at >>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1102) >>> at >>> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) >>> at >>> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1102) >>> at >>> org.apache.hadoop.hive.ql.exec.tez.DagUtils.getDefaultDestDir(DagUtils.java:638) >>> at >>> org.apache.hadoop.hive.ql.exec.tez.DagUtils.getHiveJarDirectory(DagUtils.java:728) >>> at >>> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.createJarLocalResource(TezSessionState.java:314) >>> at >>> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:169) >>> at >>> org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:153) >>> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) >>> at >>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) >>> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1504) >>> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1271) >>> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1089) >>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:912) >>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902) >>> at >>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268) >>> at >>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) >>> at >>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) >>> at >>> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792) >>> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) >>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) >>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown >>> Source) >>> at java.lang.reflect.Method.invoke(Unknown Source) >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:212) >>> >>> Am I doing something wrong? The Tez I am running by setting* >>> hive.execution.engine=mr;* but setting >>> >>> *mapreduce.framework.name <http://mapreduce.framework.name>=yarn-tez. is >>> it correct? Does it launches Tez job or is it MR? * >>> >>> Thanks and Regards, >>> >>> Archit Thakur. >>> >>> CONFIDENTIALITY NOTICE >>> NOTICE: This message is intended for the use of the individual or entity >>> to which it is addressed and may contain information that is confidential, >>> privileged and exempt from disclosure under applicable law. If the reader >>> of this message is not the intended recipient, you are hereby notified that >>> any printing, copying, dissemination, distribution, disclosure or >>> forwarding of this communication is strictly prohibited. If you have >>> received this communication in error, please contact the sender immediately >>> and delete it from your system. Thank You. >> >> >> > > > -- > Carter Shanklin > Director, Product Management > Hortonworks > (M): +1.650.644.8795 (T): @cshanklin <http://twitter.com/cshanklin> > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity > to which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
application_logs
Description: Binary data
