Re: Spark paragraph hangs

I mean I downloaded a binary of zeppelin, zeppelin-0.8.0-bin-all.tgz, that comes with all the interpreters (including spark 2.2.0) in-built. > On Nov 24, 2018, at 9:13 AM, Jeff Zhang > wrote: > > What do you mean 'stock spark interpreter' ? > > Nabeel Imtiaz mailto:ni

Re: Spark paragraph hangs

What do you mean 'stock spark interpreter' ? Nabeel Imtiaz 于2018年11月24日周六 上午4:54写道： > I’m using stock spark interpreter that comes with zeppelin 0.8.0. > > Locally I’ve successful run spark jobs on a local standalone cluster. > > > Nabeel > > > On Nov 24, 2018, at 00:40, JAIN, RAHUL wrote: > >

Re: Spark paragraph hangs

I’m using stock spark interpreter that comes with zeppelin 0.8.0. Locally I’ve successful run spark jobs on a local standalone cluster. Nabeel > On Nov 24, 2018, at 00:40, JAIN, RAHUL wrote: > > > Does the job work when your spark master is configured as local in zeppelin > spark interpret

Re: Spark paragraph hangs

Does the job work when your spark master is configured as local in zeppelin spark interpreter config ? If that works well then your real issue may be in your spark/yarn cluster. Also, have you tried doing a spark-submit manually for a similar job on command line pointing to the cluster ? -Ra

Re: Spark paragraph hangs

Nothing much useful. Following are the interpreter logs I can see before the job just hangs: INFO [2018-11-23 14:25:58,517] ({pool-2-thread-3} SchedulerFactory.java[jobStarted]:109) - Job 20181123-112240_1827913615 started by scheduler interpreter_34089707 INFO [2018-11-23 14:25:59,241] ({pool-

Re: Spark paragraph hangs

Is there something useful information your local spark process log? > 在 2018年11月23日，下午4:20，Nabeel Imtiaz 写道： > > Hi, > > > When I trying to even simply take first 10 lines of a file (like > ```batchData.take(10).foreach(println _)``` from sprakContext, the paragraph > hangs. > > If I ins

Spark paragraph hangs

Hi, When I trying to even simply take first 10 lines of a file (like ```batchData.take(10).foreach(println _)``` from sprakContext, the paragraph hangs. If I inspect the job in spark console, it shows the job in PENDING state. I check that I have more than enough memory in the system availab