Karthik,
You might find https://github.com/hortonworks/hdp-configuration-utils
useful.
That should help provide all the right configs for YARN, Tez & Hive.
thanks,
Arun
On Sat, Aug 16, 2014 at 4:40 PM, karthik Srivasthava <
karthiksrivasth...@gmail.com> wrote:
> I have total Memory of 48 G
I have total Memory of 48 GB with active 8 nodes.
Initially i set up
set yarn.nodemanager.resource.memory-mb=6144;
set yarn.scheduler.maximum-allocation-mb=6144;
set yarn.scheduler.minimum-allocation-mb=512;
46 Containers are created using all the space available..
Then i changed it to so that
Whats the value set in yarn-site ? Can you paste here ?
Sent from my iPhone
> On Aug 16, 2014, at 8:19 PM, "karthik Srivasthava"
> wrote:
>
> So i tried using these
>
> set yarn.nodemanager.resource.memory-mb ;
> set yarn.scheduler.maximum-allocation-mb;
>
> Even though, one yarn_tez
So i tried using these
set yarn.nodemanager.resource.memory-mb ;
set yarn.scheduler.maximum-allocation-mb;
Even though, one yarn_tez application is using all the space available..
Do you think i didnt set these settings properly or
i should set different parameters.?
Thanks in advance
Tweak parameters to control container memory allocation. Take a look at this:
http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/
> On Aug 16, 2014, at 12:58 PM, karthik Srivasthava
> wrote:
>
> Thanks manoj for ur response.
>
> But i struggle to run another application in
Limit the maximum usable memory by Yarn
Sent from my iPhone
> On Aug 16, 2014, at 12:16 PM, "karthik Srivasthava"
> wrote:
>
> I am using TEZ as Hive Execution Engine. When i run a query , Yarn Containers
> are using all the space available in node. This doesnt allow another query to
> run.
Thanks manoj for ur response.
But i struggle to run another application in same node as first yarn
application using all the available RAM .
On Sat, Aug 16, 2014 at 11:44 AM, Manoj wrote:
> Look into setting up yarn queues and quotas.
>
> > On Aug 16, 2014, at 8:15 AM, karthik Srivasthava
Look into setting up yarn queues and quotas.
> On Aug 16, 2014, at 8:15 AM, karthik Srivasthava
> wrote:
>
> I am using TEZ as Hive Execution Engine. When i run a query , Yarn Containers
> are using all the space available in node. This doesnt allow another query to
> run. Am i missing any co
I am using TEZ as Hive Execution Engine. When i run a query , Yarn
Containers are using all the space available in node. This doesnt allow
another query to run. Am i missing any configuration settings here. Please
let me know guys where the error might be.
Thanks in advance,
Karthik
