Use yarn logs -applicationId <appId> to see detail log.
r7raul1...@163.com From: Jadhav Shweta Date: 2015-02-26 15:00 To: user Subject: Re: ERROR OutOfMemoryError: Java heap space Hi, I tried running same query from Simple JDBC Client instead of Spring Batch framework its was working fine. But now when I am running it again I am getting new error Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0 2015-02-26 11:41:50,233 Stage-1 map = 0%, reduce = 0% 2015-02-26 11:42:11,975 Stage-1 map = 47%, reduce = 0%, Cumulative CPU 15.13 sec 2015-02-26 11:42:24,593 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 27.55 sec java.io.IOException: Could not find status of job:job_1424930506879_0003 at org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:294) at org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:547) at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:426) at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:136) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:72) Ended Job = job_1424930506879_0003 with exception 'java.io.IOException(Could not find status of job:job_1424930506879_0003)' FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask Job HistoryServer is running properly. Thanks Shweta Jadhav -----Vaibhav Gumashta <vgumas...@hortonworks.com> wrote: ----- To: "user@hive.apache.org" <user@hive.apache.org> From: Vaibhav Gumashta <vgumas...@hortonworks.com> Date: 02/26/2015 02:08AM Subject: Re: ERROR OutOfMemoryError: Java heap space --> Hi, auth=nosasl should be only provided in JDBC url if "hive.server2.authentication” is explicitly set to NOSASL in your hive-site. Otherwise, the jdbc client tries to open a plain socket while the the server is expecting a sasl negotiation to happen. Due to an issue with thrift, this results in OOM on client side. —Vaibhav From: Srinivas Thunga <srinivas.thu...@gmail.com> Reply-To: "user@hive.apache.org" <user@hive.apache.org> Date: Wednesday, February 25, 2015 at 3:40 AM To: "user@hive.apache.org" <user@hive.apache.org> Subject: Re: ERROR OutOfMemoryError: Java heap space Is your problem solved? Thanks & Regards, Srinivas T On Wed, Feb 25, 2015 at 4:17 PM, Srinivas Thunga <srinivas.thu...@gmail.com> wrote: Hi, You can set fetch size based certain datatype like blob, image, etc statement.setFetchSize(1000); Try this. Thanks & Regards, Srinivas T On Wed, Feb 25, 2015 at 4:10 PM, Jadhav Shweta <jadhav.shw...@tcs.com> wrote: Query is working fine if i have small data set in same table. But its throwing error for large data set Thanks Shweta Jadhav -----Srinivas Thunga <srinivas.thu...@gmail.com> wrote: ----- To: "user@hive.apache.org" <user@hive.apache.org> From: Srinivas Thunga <srinivas.thu...@gmail.com> Date: 02/25/2015 03:45PM Subject: Re: ERROR OutOfMemoryError: Java heap space not required. Make sure there will not be unnecessary looping in your code Thanks & Regards, Srinivas T On Wed, Feb 25, 2015 at 2:58 PM, Jadhav Shweta <jadhav.shw...@tcs.com> wrote: Do I need to increase HADOOP_HEAPSIZE I haven't set it. I have cluster with 4 machines each having 8GB ram. I read somewhere that we should mention auth=noSasl in hive jdbc url. Is it necessary Thanks Shweta Jadhav -----Jadhav Shweta <jadhav.shw...@tcs.com> wrote: ----- To: user@hive.apache.org From: Jadhav Shweta <jadhav.shw...@tcs.com> Date: 02/25/2015 02:54PM Subject: Re: ERROR OutOfMemoryError: Java heap space I am running the query using spring batch framework. Thanks Shweta Jadhav -----Srinivas Thunga <srinivas.thu...@gmail.com> wrote: ----- To: "user@hive.apache.org" <user@hive.apache.org> From: Srinivas Thunga <srinivas.thu...@gmail.com> Date: 02/25/2015 02:43PM Subject: Re: ERROR OutOfMemoryError: Java heap space Hi, Might be problem in your code. Generally it happens when there are unnecessary looping. Please check your java code. Thanks & Regards, Srinivas T On Wed, Feb 25, 2015 at 2:38 PM, Jadhav Shweta <jadhav.shw...@tcs.com> wrote: Hi I have installed apache hive 0.13.0. I have configured metastore as postgres db. Query is working fine in beeline command line interface but giving Heap space error while executing using JDBC java client. Thanks Shweta Jadhav -----Srinivas Thunga <srinivas.thu...@gmail.com> wrote: ----- To: "user@hive.apache.org" <user@hive.apache.org> From: Srinivas Thunga <srinivas.thu...@gmail.com> Date: 02/25/2015 02:32PM Subject: Re: ERROR OutOfMemoryError: Java heap space Hi, Let me know how you configured Hive? R u using cloudera? If you are installed Hive separately then have you configured Metastore? Thanks & Regards, Srinivas T On Wed, Feb 25, 2015 at 2:27 PM, Jadhav Shweta <jadhav.shw...@tcs.com> wrote: Hi I am running simple select query select * from table; thanks Shweta Jadhav -----Srinivas Thunga <srinivas.thu...@gmail.com> wrote: ----- To: "user@hive.apache.org" <user@hive.apache.org> From: Srinivas Thunga <srinivas.thu...@gmail.com> Date: 02/25/2015 01:09PM Subject: Re: Hi, Can you place the query as well which you are trying? Thanks & Regards, Srinivas T On Wed, Feb 25, 2015 at 1:02 PM, Jadhav Shweta <jadhav.shw...@tcs.com> wrote: Hi I am trying to run hive query Its getting executed from beeline interface but its throwing java.lang.OutOfMemoryError: Java heap space error when connecting using jdbc. I am using hive 0.13.0 version and hiveserver2. which parameters i need to configure for the same. thanks Shweta Jadhav =====-----=====-----===== Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you
