Hi You can first execute a simple statement in spark, through sparksql, to see if it can run normally in YARN. If sparksql is running without problems, check the zeppelin and spark on yarn issues.
Also, what do you use for zeppelin-0.7.4? zeppelin-0.8.2? Is it a branch that you maintain yourself? > 在 2019年3月15日，上午6:31，Dave Boyd <db...@incadencecorp.com> 写道： > > All: > > I have some code that worked fine in Zeppelin 0.7.4 but I am having issues > in 0.8.2 when going from spark master of local to yarn-client. Yarn client > worked in 0.7.4. > When my master is set to local[*] it runs just fine. However, as soon as I > switch to yarn-client I get the Cannot call methods on a stopped SparkContext > error. In looking at my yarn logs everything creates fine and the job > finishes without an error. The executors start just fine > from what I get out of yarn logs. > Any suggestions on where to look? This happens with any note that trys to > run spark. > > If I try this very simple code: > > // Spark Version > spark.version > I get this error: > > >> >> java.lang.IllegalStateException: Spark context stopped while waiting for >> backend at >> org.apache.spark.scheduler.TaskSchedulerImpl.waitBackendReady(TaskSchedulerImpl.scala:614) >> at >> org.apache.spark.scheduler.TaskSchedulerImpl.postStartHook(TaskSchedulerImpl.scala:169) >> at org.apache.spark.SparkContext.<init>(SparkContext.scala:567) at >> org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2313) at >> org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868) >> at >> org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860) >> at scala.Option.getOrElse(Option.scala:121) at >> org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:498) at >> org.apache.zeppelin.spark.BaseSparkScalaInterpreter.spark2CreateContext(BaseSparkScalaInterpreter.scala:259) >> at >> >> org.apache.zeppelin.spark.BaseSparkScalaInterpreter.createSparkContext(BaseSparkScalaInterpreter.scala:178) >> at >> org.apache.zeppelin.spark.SparkScala211Interpreter.open(SparkScala211Interpreter.scala:89) >> at >> org.apache.zeppelin.spark.NewSparkInterpreter.open(NewSparkInterpreter.java:102) >> at >> org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:62) at >> org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69) >> at >> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:616) >> at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at >> org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:140) at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at >> java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >> at java.lang.Thread.run(Thread.java:748) >> > What am I missing? > -- > ========= mailto:db...@incadencecorp.com <mailto:db...@incadencecorp.com> > ============ > David W. Boyd > VP, Data Solutions > 10432 Balls Ford, Suite 240 > Manassas, VA 20109 > office: +1-703-552-2862 > cell: +1-703-402-7908 > ============== http://www.incadencecorp.com/ <http://www.incadencecorp.com/> > ============ > ISO/IEC JTC1 WG9, editor ISO/IEC 20547 Big Data Reference Architecture > Chair ANSI/INCITS TC Big Data > Co-chair NIST Big Data Public Working Group Reference Architecture > First Robotic Mentor - FRC, FTC - www.iliterobotics.org > <http://www.iliterobotics.org/> > Board Member- USSTEM Foundation - www.usstem.org <http://www.usstem.org/> > > The information contained in this message may be privileged > and/or confidential and protected from disclosure. > If the reader of this message is not the intended recipient > or an employee or agent responsible for delivering this message > to the intended recipient, you are hereby notified that any > dissemination, distribution or copying of this communication > is strictly prohibited. If you have received this communication > in error, please notify the sender immediately by replying to > this message and deleting the material from any computer. > >
