Hi Shrikant.
I think perhaps this belongs in the spark user email list, not the dev email
list.
That being said, is the root cause perhaps that the k8s pod is shut down? Pods
in k8s are ephemeral and might be shut down at any time (and the containers
therein restarted in a new pod). This is esp
I have gone through debug logs of jobs. There are no failures or exceptions
in logs.
This issue does not seem to be specific to jobs as several of our jobs have
been impacted by this issue and these same jobs pass also on retry.
I am trying to figure out why the driver pod is getting deleted when
