Re: Livy failure

2017-06-26 Thread Иван Шаповалов
Got it, thank you. 2017-06-26 14:58 GMT+03:00 Jeff Zhang : > > The error message is clear, you didn't set the right configuration. You > enable the dynamic allocation, and set max executor as 2 but you set the > initial executor as 3 > > > > Caused by: java.lang.IllegalArgumentException: requirem

Re: Livy failure

2017-06-26 Thread Jeff Zhang
The error message is clear, you didn't set the right configuration. You enable the dynamic allocation, and set max executor as 2 but you set the initial executor as 3 Caused by: java.lang.IllegalArgumentException: requirement failed: initial executor number 3 must between min executor number 1 a

Re: Livy failure

2017-06-26 Thread Иван Шаповалов
Here is the failure from resource manager 17/06/26 05:55:43 ERROR ApplicationMaster: Uncaught exception: org.apache.spark.SparkException: Exception thrown in awaitResult: at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:194) at org.apache.spark.deploy.yarn.Appli

Re: Livy failure

2017-06-26 Thread Jeff Zhang
Could you check the yarn app log ? Иван Шаповалов 于2017年6月26日周一 下午6:40写道: > Hi all, > I am trying to connect and run note via livy > > *created interpreter setting:* > livy.spark.driver.cores 1 > livy.spark.driver.memory 1g > livy.spark.dynamicAllocation.cachedExecutorIdleTimeout 600 > livy.spa

Livy failure

2017-06-26 Thread Иван Шаповалов
Hi all, I am trying to connect and run note via livy *created interpreter setting:* livy.spark.driver.cores 1 livy.spark.driver.memory 1g livy.spark.dynamicAllocation.cachedExecutorIdleTimeout 600 livy.spark.dynamicAllocation.enabled true livy.spark.dynamicAllocation.initialExecutors 1 livy.spark.