Re: Stopping StreamingContext

+1 Regards, Dhaval Modi dhavalmod...@gmail.com On 29 March 2018 at 19:57, Sidney Feiner wrote: > Hey, > > I have a Spark Streaming application processing some events. > > Sometimes, I want to stop the application if a get a specific event. I > collect the executor's results in the driver and ba

Stopping StreamingContext

Hey, I have a Spark Streaming application processing some events. Sometimes, I want to stop the application if a get a specific event. I collect the executor's results in the driver and based on those results, I kill the StreamingContext using StreamingContext.stop(stopSparkContext=true). When I

Re: Stopping StreamingContext before receiver has started

This is a known race condition - root cause of SPARK-5681 On Mon, Jul 13, 2015 at 3:35 AM, Juan Rodríguez Hortalá < juan.rodriguez.hort...@gmail.com> wrote: > Hi, > > I have noticed that when StreamingContext.stop is called when no receiver > has

Stopping StreamingContext before receiver has started

Hi, I have noticed that when StreamingContext.stop is called when no receiver has started yet, then the context is not really stopped. Watching the logs it looks like a stop signal is sent to 0 receivers, because the receivers have not started yet, and then the receivers are started and the contex

Re: Stopping StreamingContext does not kill receiver

en do a ssc.stop(false). >> >> < >> http://apache-spark-user-list.1001560.n3.nabble.com/file/n11595/Screen_Shot_2014-08-06_at_4.png >> > >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-lis

Re: Stopping StreamingContext does not kill receiver

t; . > > Then do a ssc.stop(false). > > < > http://apache-spark-user-list.1001560.n3.nabble.com/file/n11595/Screen_Shot_2014-08-06_at_4.png > > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Stopping-StreamingContext

Re: Stopping StreamingContext does not kill receiver

- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Stopping-StreamingContext-does-not-kill-receiver-tp9522p11595.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --

Re: Stopping StreamingContext does not kill receiver

sing the > socketTextStream receiver. The ReceiverTracker task sticks around after a > ssc.stop(false). > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Stopping-StreamingContext-does-not-kill-receiver-tp9522p11587.html > Sent from the

Re: Stopping StreamingContext does not kill receiver

I'm running on spark 1.0.0 and I see a similar problem when using the socketTextStream receiver. The ReceiverTracker task sticks around after a ssc.stop(false). -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Stopping-StreamingContext-does-not-kill-rec

Re: Stopping StreamingContext does not kill receiver

gt; ssc.stop(stopSparkContext >>> = false, stopGracefully = true) and wait a bit, but the receiver seems >>> to stay active. >>> >>> Is this expected? I’m running 1.0.1 on EC2. >>> >>> Nick >>> ​ >>> >>>

Re: Stopping StreamingContext does not kill receiver

aming/ >> and see that I have an active Twitter receiver. Then I call >> ssc.stop(stopSparkContext >> = false, stopGracefully = true) and wait a bit, but the receiver seems >> to stay active. >> >> Is this expected? I’m running 1.0.1 on EC2. >> >> Ni

Re: Stopping StreamingContext does not kill receiver

active Twitter receiver. Then I call > ssc.stop(stopSparkContext > = false, stopGracefully = true) and wait a bit, but the receiver seems to > stay active. > > Is this expected? I’m running 1.0.1 on EC2. > > Nick > ​ > > ------ > View this messag

Stopping StreamingContext does not kill receiver

tay active. Is this expected? I’m running 1.0.1 on EC2. Nick ​ -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Stopping-StreamingContext-does-not-kill-receiver-tp9522.html Sent from the Apache Spark User List mailing list archive at Nabble.com.