Re: Resource allocation on YARN

2020-02-25 Thread Abhishek S
Hey Malcolm, **YARN** *yarn.nodemanager.resource.memory-mb (Amount of physical memory, in MB, that can be allocated for containers)* The value for this depends on if there are any other side-car applications on the machine that the node-manager runs on. eg. on your 32GB machine - if other apps on

[DISCUSS] 1.4 release

2020-02-25 Thread Cameron Lee
Hi all, We have made some updates to Samza, and we would like to make a Samza 1.4 release. We have been testing some of these changes internally at Linkedin, and we would like to send this thread for discussion for releasing to open source. Highlights of the release include improvements to local st

Re: Resource allocation on YARN

2020-02-25 Thread Malcolm McFarland
Hey Abhishek, I appreciate the feedback. I'm not seeing any actual errors -- our cluster has been running pretty reliably for several months now -- but we're revisiting our resource allocation, and I was wondering if anybody out there, maybe even at Confluent or LinkedIn, has advice about good rul