I believe we can use the properties like --executor-memory --total-executor-cores to configure the resources allocated for each application. But, in a multi user environment, shells and applications are being submitted by multiple users at the same time. All users are requesting resources with different properties. At times, some users are not getting resources of the cluster.
How to control resource usage in this case? Please share any best practices followed. As per my understanding, Fair scheduler can used for scheduling tasks within an application but not across multiple applications. Is this correct? Regards, Arun
