Re: Sizing the driver & executor cores and memory in Kubernetes cluster

2021-12-16 Thread Mich Talebzadeh
Following from the above, I did some tests on this with leaving 1 VCPU out of 4 VCPUS to the OS on each node (Container & executor) in three node GKE cluster. The RAM allocated to each node was 16GB. I then set the initial container AND executor (memory 10 10% of RAM) and incremented these in steps

Sizing the driver & executor cores and memory in Kubernetes cluster

2021-12-14 Thread Mich Talebzadeh
Hi, I have a three node k8s cluster (GKE) in Google cloud with E2 standard machines that have 4 GB of system memory per VCPU giving 4 VPCU and 16,384MB of RAM. An optimum sizing of the number of executors, CPU and memory allocation is important here. These are the assumptions: 1. You want to