Re: Spark Job Doesn't End on Mesos

2016-08-09 Thread Michael Gummelt
Is this a new issue? What version of Spark? What version of Mesos/libmesos? Can you run the job with debug logging turned on and attach the output? Do you see the corresponding message in the mesos master that indicates it received the teardown? On Tue, Aug 9, 2016 at 1:28 AM, Todd Leo wrote: >

Spark Job Doesn't End on Mesos

2016-08-09 Thread Todd Leo
Hi, I’m running Spark jobs on Mesos. When the job finishes, *SparkContext* is manually closed by sc.stop(). Then Mesos log shows: I0809 15:48:34.132014 11020 sched.cpp:1589] Asked to stop the driver I0809 15:48:34.132181 11277 sched.cpp:831] Stopping framework '20160808-170425-2365980426-5050-437