There isnt a clean programmatic way to kill the application running in the
driver from the executor. You will have to set up addition RPC mechanism to
explicitly send a signal from the executors to the application/driver to
quit.
On Wed, May 3, 2017 at 8:44 AM, Sidney Feiner <sidney.fei...@startapp.com>
wrote:
> Hey, I'm using connections to Elasticsearch from within my Spark Streaming
> application.
>
> I'm using Futures to maximize performance when it sends network requests
> to the ES cluster.
>
> Basically, I want my app to crash if any one of the executors fails to
> connect to ES.
>
>
>
> The exception gets catched and returned in my Future as a Failure(ex:
> NoNodeAvailableException) but when I handle it, I can't seem to kill my app.
>
> I tried using:
>
>
>
> fut andThen {
> *case **Failure*(ex: NoNodeAvailableException) =>
> *throw *ex
> }
>
> fut andThen {
> *case **Failure*(ex: NoNodeAvailableException) =>
> System.*exit*(-1)
> }
>
> fut onFailure {
> *case *ex: NoNodeAvailableException =>
> *throw *ex
> }
>
> fut onFailure {
> *case *ex: NoNodeAvailableException =>
> System.*exit*(-1)
> }
>
>
>
>
>
> But none of them seem to be killing my app. The System.exit(-1) kills my
> executor but that doesn't seem like the correct way to do it.
>
> And no matter what way I try, the driver stays alive.
>
>
>
> Is there a way to programmatically kill the application from within one of
> the workers?
>
>
>
> Thanks a lot J
>
>
>
>
>
> *Sidney Feiner* */* SW Developer
>
> M: +972.528197720 <+972%2052-819-7720> */* Skype: sidney.feiner.startapp
>
>
>
> [image: emailsignature]
>
>
>
>
>
