All the action happens in ApplicationMaster expecially in run method
Check ApplicationMaster#startUserApplication : userThread(Driver) which
invokes ApplicationMaster#finish method. You can also try System.exit in
your program
Regards,
Yogesh Mahajan,
SnappyData Inc, snappydata.io
On Thu, Jan 14, 2016 at 9:56 AM, Yogesh Mahajan <ymaha...@snappydata.io>
wrote:
> Hi Triones,
>
> Check the org.apache.spark.util.ShutdownHookManager : It adds this
> ShutDownHook when you start a StreamingContext
>
> Here is the code in StreamingContext.start()
>
> shutdownHookRef = ShutdownHookManager.addShutdownHook(
> StreamingContext.SHUTDOWN_HOOK_PRIORITY)(stopOnShutdown)
>
> Also looke at the following def in StreamingContext which actually stops
> the context from shutdown hook :
> private def stopOnShutdown(): Unit = {
> val stopGracefully =
> conf.getBoolean("spark.streaming.stopGracefullyOnShutdown", false)
> logInfo(s"Invoking stop(stopGracefully=$stopGracefully) from shutdown
> hook")
> // Do not stop SparkContext, let its own shutdown hook stop it
> stop(stopSparkContext = false, stopGracefully = stopGracefully)
> }
>
> Regards,
> Yogesh Mahajan,
> SnappyData Inc, snappydata.io
>
> On Thu, Jan 14, 2016 at 8:55 AM, Triones,Deng(vip.com) <
> triones.d...@vipshop.com> wrote:
>
>> More info
>>
>>
>>
>> I am using spark version 1.5.2
>>
>>
>>
>>
>>
>> *发件人:* Triones,Deng(vip.com) [mailto:triones.d...@vipshop.com]
>> *发送时间:* 2016年1月14日 11:24
>> *收件人:* user
>> *主题:* spark streaming context trigger invoke stop why?
>>
>>
>>
>> Hi all
>>
>> As I saw the driver log, the task failed 4 times in a stage, the
>> stage will be dropped when the input block was deleted before make use of.
>> After that the StreamingContext invoke stop. Does anyone know what kind of
>> akka message trigger the stop or which code trigger the shutdown hook?
>>
>>
>>
>>
>>
>> Thanks
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> Driver log:
>>
>>
>>
>> Job aborted due to stage failure: Task 410 in stage 215.0 failed 4 times
>>
>> [org.apache.spark.streaming.StreamingContext---Thread-0]: Invoking
>> stop(stopGracefully=false) from shutdown hook
>>
>>
>> 本电子邮件可能为保密文件。如果阁下非电子邮件所指定之收件人，谨请立即通知本人。敬请阁下不要使用、保存、复印、打印、散布本电子邮件及其内容，或将其用于其他任何目的或向任何人披露。谢谢您的合作！
>> This communication is intended only for the addressee(s) and may contain
>> information that is privileged and confidential. You are hereby notified
>> that, if you are not an intended recipient listed above, or an authorized
>> employee or agent of an addressee of this communication responsible for
>> delivering e-mail messages to an intended recipient, any dissemination,
>> distribution or reproduction of this communication (including any
>> attachments hereto) is strictly prohibited. If you have received this
>> communication in error, please notify us immediately by a reply e-mail
>> addressed to the sender and permanently delete the original e-mail
>> communication and any attachments from all storage devices without making
>> or otherwise retaining a copy.
>> 本电子邮件可能为保密文件。如果阁下非电子邮件所指定之收件人，谨请立即通知本人。敬请阁下不要使用、保存、复印、打印、散布本电子邮件及其内容，或将其用于其他任何目的或向任何人披露。谢谢您的合作！
>> This communication is intended only for the addressee(s) and may contain
>> information that is privileged and confidential. You are hereby notified
>> that, if you are not an intended recipient listed above, or an authorized
>> employee or agent of an addressee of this communication responsible for
>> delivering e-mail messages to an intended recipient, any dissemination,
>> distribution or reproduction of this communication (including any
>> attachments hereto) is strictly prohibited. If you have received this
>> communication in error, please notify us immediately by a reply e-mail
>> addressed to the sender and permanently delete the original e-mail
>> communication and any attachments from all storage devices without making
>> or otherwise retaining a copy.
>>
>
>
