Re: Running Samza with YARN Node label support

Thanks Bharath and Yang. It is clear now. On Thu, 19 Dec 2019, 04:04 Bharath Kumara Subramanian, < codin.mart...@gmail.com> wrote: > Hi Debraj, > > I forgot to call this out earlier. Some distribution of YARN doesn't > support node label and rack combination as part of the same request. If you >

Re: Running Samza with YARN Node label support

Hi Debraj, I forgot to call this out earlier. Some distribution of YARN doesn't support node label and rack combination as part of the same request. If you were to use node labels along with host affinity feature in

Re: Running Samza with YARN Node label support

Hi Debraj, To get the node label working, set the label configurations[1] pointed out by Yang in your application config. Samza will take care of embedding the node label in the resource request automatically if it notices the label configuration inside your application. Samza framework respects n

Re: Running Samza with YARN Node label support

I understood how I can assign labels to yarn nodes. But it is still not clear to me how can I specify the node label for a samza application. I am referring to the section "Specifying node label for application" in the link

Re: Running Samza with YARN Node label support

Hi Debraj Manna, The app-def in previous email is just an example where you can configure node labels. Yarn node labels is a general feature (not specific to Samza), and it depends on the configuration system your

Re: Running Samza with YARN Node label support

Thanks, Yang for replaying. Yes, my use case is almost similar. Can you let me know which app-def you are referring to? I am not able to locate yarn.am.container.label in samza-configurations . Is there any samza

Re: Running Samza with YARN Node label support

Hello Debraj, We do not have a formal documentation in open source to describe how yarn node label is used in general. In contrast, we have an example of using yarn node label to specify Samza container to run over "HDD" rather than default "SSD" nodes. Please take a look at the following guide an

Running Samza with YARN Node label support

Hi I am seeing running samza with yarn node label is resolved in 0.12. https://issues.apache.org/jira/browse/SAMZA-1013?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aall-tabpanel But I am not able to locate the relevant documentation in samza-yarn documentation