Hi, I've been running benchmarks against Spark in local mode in a long running process. I'm seeing threads leaking each time it runs a job. It doesn't matter if I recycle SparkContext constantly or have 1 context stay alive for the entire application lifetime.
I see a huge accumulation ongoing of "pool-xxxx-thread-1" threads with the creating thread "Executor task launch worker-xx" where x's are numbers. The number of leaks per launch worker varies but usually 1 to a few. Searching the Spark code the pool is created in the Executor class. It is `.shutdown()` in the stop for the executor. I've wired up logging and also tried shutdownNow() and awaitForTermination on the pools. Every seems okay there for every Executor that is called with `stop()` but I'm still not sure yet if every Executor is called as such, which I am looking into now. What I'm curious to know is if anyone has seen a similar issue? -- *Richard Marscher* Software Engineer Localytics Localytics.com <http://localytics.com/> | Our Blog <http://localytics.com/blog> | Twitter <http://twitter.com/localytics> | Facebook <http://facebook.com/localytics> | LinkedIn <http://www.linkedin.com/company/1148792?trk=tyah>
