Here is the failure from resource manager
17/06/26 05:55:43 ERROR ApplicationMaster: Uncaught exception:
org.apache.spark.SparkException: Exception thrown in awaitResult:
at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:194)
at
org.apache.spark.deploy.yarn.ApplicationMaster.runDriver(ApplicationMaster.scala:401)
at
org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:254)
at
org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$main$1.apply$mcV$sp(ApplicationMaster.scala:766)
at
org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:67)
at
org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:66)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
at
org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:66)
at
org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:764)
at
org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloudera.livy.repl.SparkContextInitializer$class.spark2CreateContext(SparkContextInitializer.scala:94)
at
com.cloudera.livy.repl.SparkContextInitializer$class.createSparkContext(SparkContextInitializer.scala:34)
at
com.cloudera.livy.repl.SparkInterpreter.createSparkContext(SparkInterpreter.scala:36)
at
com.cloudera.livy.repl.SparkInterpreter$$anonfun$start$1.apply$mcV$sp(SparkInterpreter.scala:89)
at
com.cloudera.livy.repl.SparkInterpreter$$anonfun$start$1.apply(SparkInterpreter.scala:68)
at
com.cloudera.livy.repl.SparkInterpreter$$anonfun$start$1.apply(SparkInterpreter.scala:68)
at
com.cloudera.livy.repl.AbstractSparkInterpreter.restoreContextClassLoader(AbstractSparkInterpreter.scala:256)
at
com.cloudera.livy.repl.SparkInterpreter.start(SparkInterpreter.scala:68)
at com.cloudera.livy.repl.Session$$anonfun$1.apply(Session.scala:76)
at com.cloudera.livy.repl.Session$$anonfun$1.apply(Session.scala:74)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: requirement failed:
initial executor number 3 must between min executor number 1 and max
executor number 2
at scala.Predef$.require(Predef.scala:224)
at
org.apache.spark.deploy.yarn.YarnSparkHadoopUtil$.getInitialTargetExecutorNumber(YarnSparkHadoopUtil.scala:304)
at
org.apache.spark.scheduler.cluster.YarnClusterSchedulerBackend.start(YarnClusterSchedulerBackend.scala:37)
at
org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:156)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:509)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2320)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860)
at scala.Option.getOrElse(Option.scala:121)
at
org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860)
... 19 more
17/06/26 05:55:43 INFO ApplicationMaster: Unregistering
ApplicationMaster with FAILED (diag message: User class threw
exception: java.lang.reflect.InvocationTargetException)
17/06/26 05:55:43 INFO ApplicationMaster: Deleting staging directory
thank you
2017-06-26 14:16 GMT+03:00 Jeff Zhang <zjf...@gmail.com>:
>
> Could you check the yarn app log ?
>
>
>
> Иван Шаповалов <shapovalov.iva...@gmail.com>于2017年6月26日周一 下午6:40写道：
>
>> Hi all,
>> I am trying to connect and run note via livy
>>
>> *created interpreter setting:*
>> livy.spark.driver.cores 1
>> livy.spark.driver.memory 1g
>> livy.spark.dynamicAllocation.cachedExecutorIdleTimeout 600
>> livy.spark.dynamicAllocation.enabled true
>> livy.spark.dynamicAllocation.initialExecutors 1
>> livy.spark.dynamicAllocation.maxExecutors 2
>> livy.spark.dynamicAllocation.minExecutors 1
>> livy.spark.executor.cores 1
>> livy.spark.executor.instances 3
>> livy.spark.executor.memory 1g
>> livy.spark.jars.packages
>> zeppelin.livy.concurrentSQL false
>> zeppelin.livy.displayAppInfo false
>> zeppelin.livy.keytab
>> zeppelin.livy.principal zeppelin
>> zeppelin.livy.pull_status.interval.millis 1000
>> zeppelin.livy.session.create_timeout 600
>> zeppelin.livy.spark.sql.maxResult 1000
>> zeppelin.livy.url http://correct-path:correct-port
>>
>> *and have following livy server config:*
>> livy.impersonation.enabled = false
>> livy.repl.enableHiveContext = true
>> ivy.server.csrf_protection.enabled = true
>> livy.spark.master = yarn.cluster
>>
>> *Trying to run paragraph drives me to:*
>>
>> INFO [2017-06-26 11:39:22,036] ({pool-2-thread-2}
>> BaseLivyInterprereter.java[createSession]:204) - Session 3 is in state
>> shutting_down, appId application_1498447942743_0011
>> INFO [2017-06-26 11:39:23,184] ({pool-2-thread-2}
>> BaseLivyInterprereter.java[createSession]:204) - Session 3 is in state dead,
>> appId application_1498447942743_0011
>> ERROR [2017-06-26 11:39:23,185] ({pool-2-thread-2}
>> BaseLivyInterprereter.java[createSession]:214) - Error when creating livy
>> session for user anonymous
>> org.apache.zeppelin.livy.LivyException: Session 3 is finished, appId:
>> application_1498447942743_0011, log: [YARN Diagnostics:, AM container is
>> launched, waiting for AM container to Register with RM]
>> at
>> org.apache.zeppelin.livy.BaseLivyInterprereter.createSession(BaseLivyInterprereter.java:209)
>> at
>> org.apache.zeppelin.livy.BaseLivyInterprereter.initLivySession(BaseLivyInterprereter.java:98)
>> at
>> org.apache.zeppelin.livy.BaseLivyInterprereter.open(BaseLivyInterprereter.java:80)
>> at
>> org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70)
>> at
>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:483)
>> at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
>> at
>> org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>>
>> Could anyone please help to understand what am I doing wrong ?
>> Thanks
>> --
>> Ivan Shapovalov
>> Kharkov, Ukraine
>>
>>
--
Ivan Shapovalov
Kharkov, Ukraine
