If you set SPARK_HOME in spark interpreter setting, then please remove SPARK_HOME in spark-env.sh, otherwise you do will hit weird issue.
小野圭二 <onoke...@gmail.com>于2018年3月6日周二 上午10:56写道： > Hi Jeff, > > I did set SPARK_HOME to use external spark at first. And that was fine > work on CentOS, but not on Mac. > Do you mean try it again? > > 2018-03-06 11:12 GMT+09:00 Jeff Zhang <zjf...@gmail.com>: > >> >>> Seems like it is not good chemistry between Spark and Zeppelin on >> Mac. From the reason that the consequence of removing the environment >> parameters, Zeppelin headed to own spark lib. >> >> Could you specify SPARK_HOME in your spark interpreter to use external >> spark instead of zeppelin's own spark lib ? >> >> >> >> >> 小野圭二 <onoke...@gmail.com>于2018年3月6日周二 上午10:06写道： >> >>> I wonder why, but it seems work fine, and would like to share how to fix >>> it. >>> >>> First of all, this issue happened just on MAC. >>> [THE ENV WAS] >>> macOS High Sierra v 10.13.3 >>> cpu core i5 >>> mem 8GB >>> Zeppelin 0.8.0 snapshot >>> Spark 2.2.0 >>> [ISSUE] >>> see this previous mails >>> [SOLUTION] >>> ref. >>> http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Zeppelin-0-6-2-Error-Interpreter-process-failed-td4743.html >>> According to the above ref, i removed 'SPARK_HOME', >>> 'SPARK_SUBMIT_OPTIONS' from my environment parameters and >>> conf/zeppelin-env.sh(set it as same like CentOS). Then restart zeppelin, >>> that's all. >>> [TO DO] >>> Seems like it is not good chemistry between Spark and Zeppelin on >>> Mac. From the reason that the consequence of removing the environment >>> parameters, Zeppelin headed to own spark lib. >>> Maybe should write this in the document as caution. >>> >>> >>> Cheers, >>> >>> Ono Keiji >>> >>> >>> >>> >>> 2018-03-06 9:15 GMT+09:00 小野圭二 <onoke...@gmail.com>: >>> >>>> Nothing changed. I guess this property does not effect cause of failing >>>> to call the interpreter. >>>> i challenged it as log4j.......zeppelin=DEBUG, but no way. >>>> >>>> 2018-03-05 18:24 GMT+09:00 Jeff Zhang <zjf...@gmail.com>: >>>> >>>>> >>>>> Try to add the following to log4j.properties for more log >>>>> >>>>> log4j.logger.org.apache.zeppelin.interpreter=DEBUG >>>>> >>>>> >>>>> >>>>> 小野圭二 <onoke...@gmail.com>于2018年3月5日周一 下午5:13写道： >>>>> >>>>>> Actually, there was not created the interpreter log file as said >>>>>> 'Cannot run interpreter' in the log. And it was run on local machine as >>>>>> MAC, i mean stand alone. i stopped every firewall to make it easy. And >>>>>> again,this happened just on Mac. On CentOS, it works fine. >>>>>> That makes me confusion. :-b >>>>>> >>>>>> 2018-03-05 14:06 GMT+09:00 Jeff Zhang <zjf...@gmail.com>: >>>>>> >>>>>>> >>>>>>> Could you check the interpreter log ? The issue of not be able to >>>>>>> connecting to thrift port could happen due to network issue, like >>>>>>> firewall >>>>>>> or you use VPN, etc. >>>>>>> >>>>>>> >>>>>>> >>>>>>> 小野圭二 <onoke...@gmail.com>于2018年3月5日周一 下午12:32写道： >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I show the all log on this error.There are 2 type error. The 1 is >>>>>>>> at the first run. The 2 is at the second run. Seems like could not >>>>>>>> established to Thrift callback port 51348 that was set at the first >>>>>>>> run. >>>>>>>> First of all, does it need to establish the callback port of Thrift >>>>>>>> server? >>>>>>>> The notebook script was just >>>>>>>> %pyspark >>>>>>>> print 'aaa' >>>>>>>> Any advice appreciate. >>>>>>>> >>>>>>>> 1. >>>>>>>> INFO [2018-03-05 13:11:13,914] ({qtp1651855867-13} >>>>>>>> VFSNotebookRepo.java[save]:221) - Saving note:2D6S6TB3U INFO >>>>>>>> [2018-03-05 >>>>>>>> 13:11:13,934] ({pool-2-thread-2} >>>>>>>> SchedulerFactory.java[jobStarted]:109) - >>>>>>>> Job 20180302-135047_99102973 started by scheduler >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter-shared_session >>>>>>>> INFO [2018-03-05 13:11:13,936] ({pool-2-thread-2} >>>>>>>> Paragraph.java[jobRun]:397) - run paragraph 20180302-135047_99102973 >>>>>>>> using >>>>>>>> pyspark >>>>>>>> RemoteInterpreter_org.apache.zeppelin.spark.PySparkInterpreter_shared_session >>>>>>>> INFO [2018-03-05 13:11:13,936] ({pool-2-thread-2} >>>>>>>> ManagedInterpreterGroup.java[getOrCreateInterpreterProcess]:57) - >>>>>>>> Create >>>>>>>> InterperterProcess for InterpreterGroup: 2CV1983WX:shared_process WARN >>>>>>>> [2018-03-05 13:11:14,010] ({pool-2-thread-2} >>>>>>>> InterpreterSetting.java[setupPropertiesForSparkR]:796) - sparkr.zip is >>>>>>>> not >>>>>>>> found, SparkR may not work. INFO [2018-03-05 13:11:14,017] >>>>>>>> ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:123) - >>>>>>>> Choose port 51347 for RemoteInterpreterProcess INFO [2018-03-05 >>>>>>>> 13:11:14,023] ({pool-2-thread-2} >>>>>>>> RemoteInterpreterManagedProcess.java[start]:130) - Thrift server for >>>>>>>> callback will start. Port: 51348 INFO [2018-03-05 13:11:14,049] >>>>>>>> ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:205) - >>>>>>>> Run >>>>>>>> interpreter process >>>>>>>> [/Users/onokeiji/zeppelin/zeppelin/bin/interpreter.sh, >>>>>>>> -d, /Users/onokeiji/zeppelin/zeppelin/interpreter/spark, -c, >>>>>>>> 172.16.20.10, >>>>>>>> -p, 51348, -l, /Users/onokeiji/zeppelin/zeppelin/local-repo/2CV1983WX, >>>>>>>> -g, >>>>>>>> spark] INFO [2018-03-05 13:11:14,238] ({Exec Default Executor} >>>>>>>> RemoteInterpreterManagedProcess.java[onProcessComplete]:258) - >>>>>>>> Interpreter >>>>>>>> process exited 0 ERROR [2018-03-05 13:12:14,058] ({pool-2-thread-2} >>>>>>>> Job.java[run]:184) - Job failed >>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: Cannot run >>>>>>>> interpreter at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess.start(RemoteInterpreterManagedProcess.java:220) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getOrCreateInterpreterProcess(RemoteInterpreter.java:101) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:153) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:126) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:265) >>>>>>>> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:423) at >>>>>>>> org.apache.zeppelin.scheduler.Job.run(Job.java:182) at >>>>>>>> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:307) >>>>>>>> at >>>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at >>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) >>>>>>>> at >>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >>>>>>>> at java.lang.Thread.run(Thread.java:748) ERROR [2018-03-05 >>>>>>>> 13:12:14,075] >>>>>>>> ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:2219) - Error >>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: Cannot run >>>>>>>> interpreter at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess.start(RemoteInterpreterManagedProcess.java:220) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getOrCreateInterpreterProcess(RemoteInterpreter.java:101) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:153) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:126) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:265) >>>>>>>> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:423) at >>>>>>>> org.apache.zeppelin.scheduler.Job.run(Job.java:182) at >>>>>>>> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:307) >>>>>>>> at >>>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at >>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) >>>>>>>> at >>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >>>>>>>> at java.lang.Thread.run(Thread.java:748) WARN [2018-03-05 13:12:14,075] >>>>>>>> ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:2227) - Job >>>>>>>> 20180302-135047_99102973 is finished, status: ERROR, exception: >>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: Cannot run >>>>>>>> interpreter, result: Cannot run interpreter INFO [2018-03-05 >>>>>>>> 13:12:14,103] >>>>>>>> ({pool-2-thread-2} VFSNotebookRepo.java[save]:221) - Saving >>>>>>>> note:2D6S6TB3U >>>>>>>> INFO [2018-03-05 13:12:14,109] ({pool-2-thread-2} >>>>>>>> SchedulerFactory.java[jobFinished]:115) - Job 20180302-135047_99102973 >>>>>>>> finished by scheduler >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter-shared_session >>>>>>>> >>>>>>>> 2. >>>>>>>> INFO [2018-03-05 13:15:32,736] ({qtp1651855867-13} >>>>>>>> VFSNotebookRepo.java[save]:221) - Saving note:2D6S6TB3U >>>>>>>> INFO [2018-03-05 13:15:32,751] ({pool-2-thread-2} >>>>>>>> SchedulerFactory.java[jobStarted]:109) - Job 20180302-135047_99102973 >>>>>>>> started by scheduler >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter-shared_session >>>>>>>> INFO [2018-03-05 13:15:32,753] ({pool-2-thread-2} >>>>>>>> Paragraph.java[jobRun]:397) - run paragraph 20180302-135047_99102973 >>>>>>>> using >>>>>>>> pyspark >>>>>>>> RemoteInterpreter_org.apache.zeppelin.spark.PySparkInterpreter_shared_session >>>>>>>> ERROR [2018-03-05 13:15:32,882] ({pool-2-thread-2} >>>>>>>> Job.java[run]:184) - Job failed >>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>> org.apache.thrift.transport.TTransportException: >>>>>>>> java.net.ConnectException: Connection refused (Connection refused) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:152) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:154) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:126) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:265) >>>>>>>> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:423) >>>>>>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:182) >>>>>>>> at >>>>>>>> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:307) >>>>>>>> at >>>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>>>>>>> at >>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) >>>>>>>> at >>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >>>>>>>> at java.lang.Thread.run(Thread.java:748) >>>>>>>> Caused by: org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>> org.apache.thrift.transport.TTransportException: >>>>>>>> java.net.ConnectException: Connection refused (Connection refused) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37) >>>>>>>> at >>>>>>>> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60) >>>>>>>> at >>>>>>>> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861) >>>>>>>> at >>>>>>>> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435) >>>>>>>> at >>>>>>>> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:73) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:144) >>>>>>>> ... 13 more >>>>>>>> Caused by: org.apache.thrift.transport.TTransportException: >>>>>>>> java.net.ConnectException: Connection refused (Connection refused) >>>>>>>> at org.apache.thrift.transport.TSocket.open(TSocket.java:187) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51) >>>>>>>> ... 20 more >>>>>>>> Caused by: java.net.ConnectException: Connection refused >>>>>>>> (Connection refused) >>>>>>>> at java.net.PlainSocketImpl.socketConnect(Native Method) >>>>>>>> at java.net >>>>>>>> .AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) >>>>>>>> at java.net >>>>>>>> .AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) >>>>>>>> at java.net >>>>>>>> .AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) >>>>>>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) >>>>>>>> at java.net.Socket.connect(Socket.java:589) >>>>>>>> at org.apache.thrift.transport.TSocket.open(TSocket.java:182) >>>>>>>> ... 21 more >>>>>>>> ERROR [2018-03-05 13:15:32,884] ({pool-2-thread-2} >>>>>>>> NotebookServer.java[afterStatusChange]:2219) - Error >>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>> org.apache.thrift.transport.TTransportException: >>>>>>>> java.net.ConnectException: Connection refused (Connection refused) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:152) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:154) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:126) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:265) >>>>>>>> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:423) >>>>>>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:182) >>>>>>>> at >>>>>>>> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:307) >>>>>>>> at >>>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>>>>>>> at >>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) >>>>>>>> at >>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >>>>>>>> at java.lang.Thread.run(Thread.java:748) >>>>>>>> Caused by: org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>> org.apache.thrift.transport.TTransportException: >>>>>>>> java.net.ConnectException: Connection refused (Connection refused) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37) >>>>>>>> at >>>>>>>> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60) >>>>>>>> at >>>>>>>> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861) >>>>>>>> at >>>>>>>> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435) >>>>>>>> at >>>>>>>> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:73) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:144) >>>>>>>> ... 13 more >>>>>>>> Caused by: org.apache.thrift.transport.TTransportException: >>>>>>>> java.net.ConnectException: Connection refused (Connection refused) >>>>>>>> at org.apache.thrift.transport.TSocket.open(TSocket.java:187) >>>>>>>> at >>>>>>>> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51) >>>>>>>> ... 20 more >>>>>>>> Caused by: java.net.ConnectException: Connection refused >>>>>>>> (Connection refused) >>>>>>>> at java.net.PlainSocketImpl.socketConnect(Native Method) >>>>>>>> at java.net >>>>>>>> .AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) >>>>>>>> at java.net >>>>>>>> .AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) >>>>>>>> at java.net >>>>>>>> .AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) >>>>>>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) >>>>>>>> at java.net.Socket.connect(Socket.java:589) >>>>>>>> at org.apache.thrift.transport.TSocket.open(TSocket.java:182) >>>>>>>> ... 21 more >>>>>>>> WARN [2018-03-05 13:15:32,885] ({pool-2-thread-2} >>>>>>>> NotebookServer.java[afterStatusChange]:2227) - Job >>>>>>>> 20180302-135047_99102973 >>>>>>>> is finished, status: ERROR, exception: >>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>> org.apache.thrift.transport.TTransportException: >>>>>>>> java.net.ConnectException: Connection refused (Connection refused), >>>>>>>> result: >>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>> org.apache.thrift.transport.TTransportException: >>>>>>>> java.net.ConnectException: Connection refused (Connection refused) >>>>>>>> INFO [2018-03-05 13:15:32,910] ({pool-2-thread-2} >>>>>>>> VFSNotebookRepo.java[save]:221) - Saving note:2D6S6TB3U >>>>>>>> INFO [2018-03-05 13:15:32,914] ({pool-2-thread-2} >>>>>>>> SchedulerFactory.java[jobFinished]:115) - Job 20180302-135047_99102973 >>>>>>>> finished by scheduler >>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter-shared_session >>>>>>>> >>>>>>>> 2018-03-02 15:07 GMT+09:00 小野圭二 <onoke...@gmail.com>: >>>>>>>> >>>>>>>>> Hi Jeff, >>>>>>>>> >>>>>>>>> No one graved 63039 so far on Mac. >>>>>>>>> Yes, this issue has been reproduced as changing the port NO >>>>>>>>> randomly. >>>>>>>>> >>>>>>>>> Ono >>>>>>>>> >>>>>>>>> >>>>>>>>> 2018-03-02 14:49 GMT+09:00 Jeff Zhang <zjf...@gmail.com>: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> It is possible that 63039 is used by other processes, as Mac can >>>>>>>>>> not detect whether this port is used or not from my experience. >>>>>>>>>> >>>>>>>>>> And can you reproduce this issue consistently ? Because each time >>>>>>>>>> zeppelin will chose a random port as long as you don't specify the >>>>>>>>>> portRange, so if you reproduce it consistently, it might due to other >>>>>>>>>> issues. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 小野圭二 <onoke...@gmail.com>于2018年3月2日周五 下午1:32写道： >>>>>>>>>> >>>>>>>>>>> Hi there, >>>>>>>>>>> >>>>>>>>>>> I am so confusing now, let me know why if You know this, please. >>>>>>>>>>> >>>>>>>>>>> The env: >>>>>>>>>>> Mac: High Sierra ver.10.13.3 >>>>>>>>>>> Zep: 0.7.3 >>>>>>>>>>> Spark: 2.2.0 >>>>>>>>>>> Java: 1.8 >>>>>>>>>>> notebook script: %pyspark print 'aaa' (just >>>>>>>>>>> it for test) >>>>>>>>>>> >>>>>>>>>>> In this env, i got the error in log as >>>>>>>>>>> >>>>>>>>>>> ERROR ({pool-2-thread-2} Job.java[run]:184) - Job failed >>>>>>>>>>> >>>>>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>>>>> org.apache.thrift.transport.TTransportException: >>>>>>>>>>> java.net.ConnectException: Connection refused (Connection refused) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Indeed, before running notebook, >>>>>>>>>>> RemoteInterpreterManagedProcess.java prepares a Port for callback >>>>>>>>>>> to Thrift. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> INFO ({pool-2-thread-1} >>>>>>>>>>> RemoteInterpreterManagedProcess.java[start]:130) - Thrift server for >>>>>>>>>>> callback will start. Port: 63039 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I guess, the reason why the error was Zeppelin could not attache >>>>>>>>>>> to the Thrift PORT. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> And my confusion is, when i run the same module on CentOS, it >>>>>>>>>>> worked fine. >>>>>>>>>>> CentOS: 7 >>>>>>>>>>> Zep: same >>>>>>>>>>> Spark: same >>>>>>>>>>> Java: same >>>>>>>>>>> notebook script: same >>>>>>>>>>> >>>>>>>>>>> And there was no the Thift port description in the log file. >>>>>>>>>>> From this consequence, the fine working was cause of did not use >>>>>>>>>>> Thrift >>>>>>>>>>> port on CentOS. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Do You know actually there is different behavior between Mac and >>>>>>>>>>> CentOS? And i am so appreciated if You know the workaround on Mac. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Best, >>>>>>>>>>> Ono >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>> >>> >
