Hi,
I think that this is more a Yarn question than a Samza one, but I am really
stuck on this. I am having problems understanding some (I belive) basic
concepts on resource management, and so I cannot have things working.
In have two virtual machines with 2 cpus and 2048MB ram each. I have Debian
installed on them and Samza over it. I am trying to get three samza jobs
working on my cluster, and I expect that I will have to run some more in the
future (at least two).
When I start Yarn I see in the web administration that I am offered a total
amount of 16Gb of RAM and 16 Vcores, which does not make sense to me on what I
have assigned.
I have configured yarn-site.xml with the following:
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>128</value>
</property>
<property>
<name>yarn.scheduler.maximum-allocation-mb</name>
<value>2048</value>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-vcores</name>
<value>1</value>
</property>
<property>
<name>yarn.scheduler.maximum-allocation-vcores</name>
<value>2</value>
</property>
I changed the capacity scheduler.xml in this value (from 0.1)
<property>
<name>yarn.scheduler.capacity.maximum-am-resource-percent</name>
<value>0.5</value>
</property>
The three jobs have the following
yarn.container.memory.mb=256
yarn.am.container.memory.mb=256
When I launch the jobs I can only get one in the running state. In fact, and
that surprises me, even when I launch them individually only one of them gets
to RUNNING. In this moment I have no way of running two of them. I have tried
to move the values above but I got no result in different combinations.
I have not seen any error in the logs.
What is preventing the jobs from getting to RUNNING state?
Thanks.
Jordi
________________________________
Jordi Blasi Uribarri
Área I+D+i
[email protected]
Oficina Bilbao
[http://www.nextel.es/wp-content/uploads/Firma_Nextel_2015.png]
