Hey folks, Does anybody have recommendations for resource allocation configs when running Samza on YARN? Ie, for a box that has 32GB of memory and 4 CPUs -- and let's say we're running a Samza task with 1000 partitions -- any suggestions on what to set for:
*YARN* yarn.nodemanager.resource.memory-mb yarn.nodemanager.resource.cpu-vcores yarn.nodemanager.resource.percentage-physical-cpu-limit *SAMZA* cluster-manager.container.memory.mb cluster-manager.container.cpu.cores yarn.am.container.memory.mb task.opts yarn.am.opts job.container.count job.container.thread.pool.size Also, do you recommend scaling up in box YARN node processing capability, or out in YARN node count? Thanks, Malcolm McFarland Cavulus This correspondence is from HealthPlanCRM, LLC, d/b/a Cavulus. Any unauthorized or improper disclosure, copying, distribution, or use of the contents of this message is prohibited. The information contained in this message is intended only for the personal and confidential use of the recipient(s) named above. If you have received this message in error, please notify the sender immediately and delete the original message.
