Re: Spark on K8s : property simillar to yarn.max.application.attempt

2022-02-04 Thread Pablo Langa Blanco
Hi Pralabh, I understand that this is a question more related to K8s than to Spark itself. In K8s you have a set of objects to control the state of your pods, which allow you, among other things, to redeploy pods that have failed. This would apply to the pod containing the driver container. https:

Re: Spark on K8s : property simillar to yarn.max.application.attempt

2022-02-04 Thread Mich Talebzadeh
Not as far as I know. If your driver pod fails, then you need to resubmit the job. I cannot see what else can be done? HTH view my Linkedin profile *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, da

Spark on K8s : property simillar to yarn.max.application.attempt

2022-02-04 Thread Pralabh Kumar
Hi Spark Team I am running spark on K8s and looking for a property/mechanism similar to yarn.max.application.attempt . I know this is not really a spark question , but i thought if anyone have faced the similar issue, Basically I want if my driver pod fails , it should be retried on a different