Help Needed to handle Hive Error: Container [xxx] is running beyond physical memory limits.

2019-11-14 Thread Sai Teja Desu
Hello All, I'm new to hive development and I'm memory limitation error for running a simple query with a predicate which should return only a few records. Below are the details of the Hive table, Query and Error. Please advise me on how to efficiently query on predicates which does not have partit

Re: Help Needed to handle Hive Error: Container [xxx] is running beyond physical memory limits.

2019-11-14 Thread Venkatesh Selvaraj
Try increasing the AM Container memory. set it to 2 gigs may be. Regards, Venkat On Thu, Nov 14, 2019, 6:46 AM Sai Teja Desu < saiteja.d...@globalfoundries.com> wrote: > Hello All, > > I'm new to hive development and I'm memory limitation error for running a > simple query with a predicate which

Re: Help Needed to handle Hive Error: Container [xxx] is running beyond physical memory limits.

2019-11-14 Thread Sai Teja Desu
Thanks for the reply Venkatesh. I did tried to increase the tez container size to 4GB but still giving me the same error. In addition, below are the settings I have tried: set mapreduce.map.memory.mb=4096; set mapreduce.map.java.opts=-Xmx3686m; set mapreduce.reduce.memory.mb=8192; set mapreduce.

Re: Help Needed to handle Hive Error: Container [xxx] is running beyond physical memory limits.

2019-11-14 Thread Pau Tallada
Hi, The error is from the AM (Application Master), because it has s many partitions to orchestrate that needs lots of RAM. As Venkat said, try increasing tez.am.resource.memory.mb to 2G, even 4 or 8 might be needed. Cheers, Pau. Missatge de Sai Teja Desu del dia dj., 14 de nov. 2019 a

What is the Hive HA processing mechanism?

2019-11-14 Thread qq
Hello:       What is the Hive HA processing mechanism? What problems can be solved?       thinks. I am looking forward to your reply??