Some additional information: the main.process shares jar files with the Spark job's driver and executor as their classpaths. It could not be those files' read/write lock, right?
---????????--- ??????: "Ted Yu"<[email protected]> ????????: 2015??10??30?? 11:46:21 ??????: "??????"<[email protected]>; ????: "jey"<[email protected]>;"user"<[email protected]>; ????: Re: SparkLauncher is blocked until main process is killed. Not much clue from the snippet on screen. Is it possible to pastebin the whole jstack output ? On Thu, Oct 29, 2015 at 7:58 PM, ?????? <[email protected]> wrote: ???????? Here's a part of the jstack output. The release is 1.5.1. ---??????件--- ??件??: "Ted Yu"<[email protected]> ????????: 2015??10??30?? 10:11:34 ??件??: "jey"<[email protected]>; ????: "??????"<[email protected]>;"user"<[email protected]>; ????: Re: SparkLauncher is blocked until main process is killed. Which Spark release are you using ? Please note the typo in email subject (corrected as of this reply) On Thu, Oct 29, 2015 at 7:00 PM, Jey Kottalam <[email protected]> wrote: Could you please provide the jstack output? That would help the devs identify the blocking operation more easily. On Thu, Oct 29, 2015 at 6:54 PM, ?????? <[email protected]> wrote: I tried to use SparkLauncher (org.apache.spark.launcher.SparkLauncher) to submit a Spark Streaming job, however, in my test, the SparkSubmit process got stuck in the "addJar" procedure. Only when the main process (the caller of SparkLauncher) is killed, the submit procedeure continues to run. I ran jstack for the process, it seems jetty was blocking it, and I'm pretty sure there was no port conflicts. The environment is RHEL(RedHot Enterprise Linux) 6u3 x64, Spark runs in standalone mode. Did this happen to any of you?
