@sagar - YARN kill is not a reliable process for spark streaming.
Regards,
Dhaval Modi
dhavalmod...@gmail.com
On 8 March 2018 at 17:18, bsikander wrote:
> I am running in Spark standalone mode. No YARN.
>
> anyways, yarn application -kill is a manual process. I donot want that. I
> was to pro
I am running in Spark standalone mode. No YARN.
anyways, yarn application -kill is a manual process. I donot want that. I
was to properly kill the driver/application programatically.
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
I am assuming you are running in yarn cluster mode. Have you tried yarn
application -kill application_id ?
With regards,
Sagar Grover
Phone - 7022175584
On Thu, Mar 8, 2018 at 4:03 PM, bsikander wrote:
> I have scenarios for both.
> So, I want to kill both batch and streaming midway, if require
I have scenarios for both.
So, I want to kill both batch and streaming midway, if required.
Usecase:
Normally, if everything is okay we don't kill the application but sometimes
while accessing external resources (like Kafka) something can go wrong. In
that case, the application can become useless
What do you mean by stopping applications?
Do you want to kill a batch application mid way or are you running
streaming jobs that you want to kill?
With regards,
Sagar Grover
On Thu, Mar 8, 2018 at 1:45 PM, bsikander wrote:
> Any help would be much appreciated. This seems to be a common problem
Any help would be much appreciated. This seems to be a common problem.
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
It seems to be related to this issue from Kafka
https://issues.apache.org/jira/browse/KAFKA-1894
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
