Karthik, You might find https://github.com/hortonworks/hdp-configuration-utils useful.
That should help provide all the right configs for YARN, Tez & Hive. thanks, Arun On Sat, Aug 16, 2014 at 4:40 PM, karthik Srivasthava < karthiksrivasth...@gmail.com> wrote: > I have total Memory of 48 GB with active 8 nodes. > > Initially i set up > set yarn.nodemanager.resource.memory-mb=6144; > set yarn.scheduler.maximum-allocation-mb=6144; > set yarn.scheduler.minimum-allocation-mb=512; > > > 46 Containers are created using all the space available.. > > Then i changed it to so that i can run other simple queries with remaining > 8 GB.. But it still takes all the memory. > > set yarn.nodemanager.resource.memory-mb=5120; > set yarn.scheduler.maximum-allocation-mb=5120; > set yarn.scheduler.minimum-allocation-mb=512; > > Even now 46 containers are created using all space.. > > > Please accept apologies if it doesnt make any sense.. But please tell me > where i made mistake > > > On Sat, Aug 16, 2014 at 6:21 PM, Siddharth Tiwari < > siddharth.tiw...@live.com> wrote: > >> Whats the value set in yarn-site ? Can you paste here ? >> >> Sent from my iPhone >> >> On Aug 16, 2014, at 8:19 PM, "karthik Srivasthava" < >> karthiksrivasth...@gmail.com> wrote: >> >> So i tried using these >> >> set yarn.nodemanager.resource.memory-mb ; >> set yarn.scheduler.maximum-allocation-mb; >> >> Even though, one yarn_tez application is using all the space available.. >> Do you think i didnt set these settings properly or >> i should set different parameters.? >> >> Thanks in advance >> >> >> >> >> >> >> >> >> >> On Sat, Aug 16, 2014 at 3:34 PM, Manoj <manoj.murum...@gmail.com> wrote: >> >>> Tweak parameters to control container memory allocation. Take a look at >>> this: >>> http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/ >>> >>> On Aug 16, 2014, at 12:58 PM, karthik Srivasthava < >>> karthiksrivasth...@gmail.com> wrote: >>> >>> Thanks manoj for ur response. >>> >>> But i struggle to run another application in same node as first yarn >>> application using all the available RAM . >>> >>> >>> >>> >>> >>> >>> On Sat, Aug 16, 2014 at 11:44 AM, Manoj <manoj.murum...@gmail.com> >>> wrote: >>> >>>> Look into setting up yarn queues and quotas. >>>> >>>> > On Aug 16, 2014, at 8:15 AM, karthik Srivasthava < >>>> karthiksrivasth...@gmail.com> wrote: >>>> > >>>> > I am using TEZ as Hive Execution Engine. When i run a query , Yarn >>>> Containers are using all the space available in node. This doesnt allow >>>> another query to run. Am i missing any configuration settings here. Please >>>> let me know guys where the error might be. >>>> > >>>> > Thanks in advance, >>>> > >>>> > Karthik >>>> >>> >>> >> > -- -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
