Re: CPU allocation when running YARN on Docker

2019-04-15 Thread Malcolm McFarland
Hi Prabhu, Sorry for the delayed response; it actually was the maximum-allocation-vcores setting. I interpreted the description for maximum-allocation-vcores as the per-container setting, when it actually seems to be the full allocation across the cluster. Cheers, Malcolm On Tue, Apr 2, 2019 at

Re: CPU allocation when running YARN on Docker

2019-04-02 Thread Prabhu Josephraj
Hi Malcolm, Scheduler sets the max-vcores in RegisterApplicationMasterResponse from the queue's configured max value (yarn.scheduler.capacity.root..maximum-allocation-vcores) from Scheduler Configuration (capacity- scheduler.xml / fair-scheduler.xml). If not specified, returns the value from

CPU allocation when running YARN on Docker

2019-04-01 Thread Malcolm McFarland
Hey folks, (apologies if this is a duplicate, I don't think my first message went through) I'm running Samza 0.14.1 with YARN 2.6.1 on Docker 18.06.1 in ECS. (I know YARN on Docker is somewhat unorthodox, but it's how the ops team at our company has things setup.) It's running quite well overall