Re: spark, autoscaling and handling node loss with autoscaling

I did some tests on a three node Dataproc cluster with autoscaling on. One master node and 2 work nodes. the master node was called ctpcluster-m and the worker nodes were ctpcluster-w-0 and ctpcluster-w-1 respectively I started a spark-submit job with the following autoscaling parameters added

spark, autoscaling and handling node loss with autoscaling

This question arises when Spark is offered as a managed service on a cluster of VMs in Cloud. For example, Google Dataproc or Amazon EMR among others >From what I can see in autoscaling setup, you will always need a minimum of two w