Hi,
I'm suffering due to the error when I'm running the zeppelin script.
I think there's a problem when zeppelin tries to connect to external Spark
master.
I've set up the external Spark master with AWS Elastic Map Reduce
(yarn-client)
I've also checked that 7077 port is opened.
Here's error log in web UI:
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:190)
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:212)
org.apache.zeppelin.scheduler.Job.run(Job.java:170)
org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:296)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
java.util.concurrent.FutureTask.run(FutureTask.java:262)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
And Here's error log in
zeppelin-interpreter-spark-root-ip-172-31-17-168.log:
WARN [2015-06-09 05:33:03,073]
({sparkDriver-akka.actor.default-dispatcher-5}
Logging.scala[logWarning]:71) - Could not connect to akka.tcp://
sparkmas...@ec2-52-26-27-71.us-west-2.compute.amazonaws.com:7077:
akka.remote.InvalidAssociation: Invalid address: akka.tcp://
sparkmas...@ec2-52-26-27-71.us-west-2.compute.amazonaws.com:7077
WARN [2015-06-09 05:33:03,083]
({sparkDriver-akka.actor.default-dispatcher-2}
Slf4jLogger.scala[apply$mcV$sp]:71) - Tried to associate with unreachable
remote address [akka.tcp://
sparkmas...@ec2-52-26-27-71.us-west-2.compute.amazonaws.com:7077]. Address
is now gated for 5000 ms, all messages to this address will be delivered to
dead letters. Reason: Connection refused:
ec2-52-26-27-71.us-west-2.compute.amazonaws.com/172.31.25.181:7077
INFO [2015-06-09 05:33:23,035]
({sparkDriver-akka.actor.default-dispatcher-2} Logging.scala[logInfo]:59) -
Connecting to master akka.tcp://
sparkmas...@ec2-52-26-27-71.us-west-2.compute.amazonaws.com:7077/user/Master.
..
WARN [2015-06-09 05:33:23,043]
({sparkDriver-akka.actor.default-dispatcher-2}
Logging.scala[logWarning]:71) - Could not connect to akka.tcp://
sparkmas...@ec2-52-26-27-71.us-west-2.compute.amazonaws.com:7077:
akka.remote.InvalidAssociation: Invalid address: akka.tcp://
sparkmas...@ec2-52-26-27-71.us-west-2.compute.amazonaws.com:7077
WARN [2015-06-09 05:33:23,044]
({sparkDriver-akka.actor.default-dispatcher-2}
Slf4jLogger.scala[apply$mcV$sp]:71) - Tried to associate with unreachable
remote address [akka.tcp://
sparkmas...@ec2-52-26-27-71.us-west-2.compute.amazonaws.com:7077]. Address
is now gated for 5000 ms, all messages to this address will be delivered to
dead letters. Reason: Connection refused:
ec2-52-26-27-71.us-west-2.compute.amazonaws.com/172.31.25.181:7077
INFO [2015-06-09 05:33:43,033]
({sparkDriver-akka.actor.default-dispatcher-2} Logging.scala[logInfo]:59) -
Connecting to master akka.tcp://
sparkmas...@ec2-52-26-27-71.us-west-2.compute.amazonaws.com:7077/user/Master.
..
WARN [2015-06-09 05:33:43,043]
({sparkDriver-akka.actor.default-dispatcher-4}
Logging.scala[logWarning]:71) - Could not connect to akka.tcp://
sparkmas...@ec2-52-26-27-71.us-west-2.compute.amazonaws.com:7077:
akka.remote.InvalidAssociation: Invalid address: akka.tcp://
sparkmas...@ec2-52-26-27-71.us-west-2.compute.amazonaws.com:7077
WARN [2015-06-09 05:33:43,043]
({sparkDriver-akka.actor.default-dispatcher-5}
Slf4jLogger.scala[apply$mcV$sp]:71) - Tried to associate with unreachable
remote address [akka.tcp://
sparkmas...@ec2-52-26-27-71.us-west-2.compute.amazonaws.com:7077]. Address
is now gated for 5000 ms, all messages to this address will be delivered to
dead letters. Reason: Connection refused:
ec2-52-26-27-71.us-west-2.compute.amazonaws.com/172.31.25.181:7077
ERROR [2015-06-09 05:34:03,034]
({sparkDriver-akka.actor.default-dispatcher-5} Logging.scala[logError]:75)
- Application has been killed. Reason: All masters are unresponsive! Giving
up.
WARN [2015-06-09 05:34:03,035] ({pool-2-thread-3}
Logging.scala[logWarning]:71) - Application ID is not initialized yet.
ERROR [2015-06-09 05:34:03,035]
({sparkDriver-akka.actor.default-dispatcher-5} Logging.scala[logError]:75)
- Exiting due to error from cluster scheduler: All masters are
unresponsive! Giving up.
Best,
Jihun Choi
