Nothing changed. I guess this property does not effect cause of failing to
call the interpreter.
i challenged it as log4j.......zeppelin=DEBUG, but no way.
2018-03-05 18:24 GMT+09:00 Jeff Zhang <zjf...@gmail.com>:
>
> Try to add the following to log4j.properties for more log
>
> log4j.logger.org.apache.zeppelin.interpreter=DEBUG
>
>
>
> 小野圭二 <onoke...@gmail.com>于2018年3月5日周一 下午5:13写道：
>
>> Actually, there was not created the interpreter log file as said 'Cannot
>> run interpreter' in the log. And it was run on local machine as MAC, i mean
>> stand alone. i stopped every firewall to make it easy. And again,this
>> happened just on Mac. On CentOS, it works fine.
>> That makes me confusion. :-b
>>
>> 2018-03-05 14:06 GMT+09:00 Jeff Zhang <zjf...@gmail.com>:
>>
>>>
>>> Could you check the interpreter log ? The issue of not be able to
>>> connecting to thrift port could happen due to network issue, like firewall
>>> or you use VPN, etc.
>>>
>>>
>>>
>>> 小野圭二 <onoke...@gmail.com>于2018年3月5日周一 下午12:32写道：
>>>
>>>> Hi,
>>>>
>>>> I show the all log on this error.There are 2 type error. The 1 is at
>>>> the first run. The 2 is at the second run. Seems like could not established
>>>> to Thrift callback port 51348 that was set at the first run.
>>>> First of all, does it need to establish the callback port of Thrift
>>>> server?
>>>> The notebook script was just
>>>> %pyspark
>>>> print 'aaa'
>>>> Any advice appreciate.
>>>>
>>>> 1.
>>>> INFO [2018-03-05 13:11:13,914] ({qtp1651855867-13}
>>>> VFSNotebookRepo.java[save]:221) - Saving note:2D6S6TB3U INFO
>>>> [2018-03-05 13:11:13,934] ({pool-2-thread-2}
>>>> SchedulerFactory.java[jobStarted]:109)
>>>> - Job 20180302-135047_99102973 started by scheduler org.apache.zeppelin.
>>>> interpreter.remote.RemoteInterpreter-shared_session INFO [2018-03-05
>>>> 13:11:13,936] ({pool-2-thread-2} Paragraph.java[jobRun]:397) - run
>>>> paragraph 20180302-135047_99102973 using pyspark
>>>> RemoteInterpreter_org.apache.zeppelin.spark.PySparkInterpreter_shared_session
>>>> INFO [2018-03-05 13:11:13,936] ({pool-2-thread-2}
>>>> ManagedInterpreterGroup.java[getOrCreateInterpreterProcess]:57) -
>>>> Create InterperterProcess for InterpreterGroup: 2CV1983WX:shared_process
>>>> WARN [2018-03-05 13:11:14,010] ({pool-2-thread-2}
>>>> InterpreterSetting.java[setupPropertiesForSparkR]:796)
>>>> - sparkr.zip is not found, SparkR may not work. INFO [2018-03-05
>>>> 13:11:14,017] ({pool-2-thread-2}
>>>> RemoteInterpreterManagedProcess.java[start]:123)
>>>> - Choose port 51347 for RemoteInterpreterProcess INFO [2018-03-05
>>>> 13:11:14,023] ({pool-2-thread-2}
>>>> RemoteInterpreterManagedProcess.java[start]:130)
>>>> - Thrift server for callback will start. Port: 51348 INFO [2018-03-05
>>>> 13:11:14,049] ({pool-2-thread-2}
>>>> RemoteInterpreterManagedProcess.java[start]:205)
>>>> - Run interpreter process
>>>> [/Users/onokeiji/zeppelin/zeppelin/bin/interpreter.sh,
>>>> -d, /Users/onokeiji/zeppelin/zeppelin/interpreter/spark, -c,
>>>> 172.16.20.10, -p, 51348, -l,
>>>> /Users/onokeiji/zeppelin/zeppelin/local-repo/2CV1983WX,
>>>> -g, spark] INFO [2018-03-05 13:11:14,238] ({Exec Default Executor}
>>>> RemoteInterpreterManagedProcess.java[onProcessComplete]:258) -
>>>> Interpreter process exited 0 ERROR [2018-03-05 13:12:14,058]
>>>> ({pool-2-thread-2} Job.java[run]:184) - Job failed org.apache.zeppelin.
>>>> interpreter.InterpreterException: Cannot run interpreter at
>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProces
>>>> s.start(RemoteInterpreterManagedProcess.java:220) at
>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.
>>>> getOrCreateInterpreterProcess(RemoteInterpreter.java:101) at
>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_
>>>> create(RemoteInterpreter.java:153) at org.apache.zeppelin.
>>>> interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:126)
>>>> at org.apache.zeppelin.interpreter.remote.
>>>> RemoteInterpreter.getFormType(RemoteInterpreter.java:265) at
>>>> org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:423) at
>>>> org.apache.zeppelin.scheduler.Job.run(Job.java:182) at
>>>> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:307)
>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
>>>> java.util.concurrent.ScheduledThreadPoolExecutor$
>>>> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$
>>>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>>> at java.lang.Thread.run(Thread.java:748) ERROR [2018-03-05
>>>> 13:12:14,075] ({pool-2-thread-2}
>>>> NotebookServer.java[afterStatusChange]:2219)
>>>> - Error org.apache.zeppelin.interpreter.InterpreterException: Cannot
>>>> run interpreter at org.apache.zeppelin.interpreter.remote.
>>>> RemoteInterpreterManagedProcess.start(RemoteInterpreterManagedProcess.java:220)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.
>>>> getOrCreateInterpreterProcess(RemoteInterpreter.java:101) at
>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_
>>>> create(RemoteInterpreter.java:153) at org.apache.zeppelin.
>>>> interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:126)
>>>> at org.apache.zeppelin.interpreter.remote.
>>>> RemoteInterpreter.getFormType(RemoteInterpreter.java:265) at
>>>> org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:423) at
>>>> org.apache.zeppelin.scheduler.Job.run(Job.java:182) at
>>>> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:307)
>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
>>>> java.util.concurrent.ScheduledThreadPoolExecutor$
>>>> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$
>>>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>>> at java.lang.Thread.run(Thread.java:748) WARN [2018-03-05
>>>> 13:12:14,075] ({pool-2-thread-2}
>>>> NotebookServer.java[afterStatusChange]:2227)
>>>> - Job 20180302-135047_99102973 is finished, status: ERROR, exception:
>>>> org.apache.zeppelin.interpreter.InterpreterException: Cannot run
>>>> interpreter, result: Cannot run interpreter INFO [2018-03-05 13:12:14,103]
>>>> ({pool-2-thread-2} VFSNotebookRepo.java[save]:221) - Saving
>>>> note:2D6S6TB3U INFO [2018-03-05 13:12:14,109] ({pool-2-thread-2}
>>>> SchedulerFactory.java[jobFinished]:115) - Job 20180302-135047_99102973
>>>> finished by scheduler org.apache.zeppelin.interpreter.remote.
>>>> RemoteInterpreter-shared_session
>>>>
>>>> 2.
>>>> INFO [2018-03-05 13:15:32,736] ({qtp1651855867-13}
>>>> VFSNotebookRepo.java[save]:221) - Saving note:2D6S6TB3U
>>>> INFO [2018-03-05 13:15:32,751] ({pool-2-thread-2}
>>>> SchedulerFactory.java[jobStarted]:109) - Job 20180302-135047_99102973
>>>> started by scheduler org.apache.zeppelin.interpreter.remote.
>>>> RemoteInterpreter-shared_session
>>>> INFO [2018-03-05 13:15:32,753] ({pool-2-thread-2}
>>>> Paragraph.java[jobRun]:397) - run paragraph 20180302-135047_99102973 using
>>>> pyspark RemoteInterpreter_org.apache.zeppelin.spark.
>>>> PySparkInterpreter_shared_session
>>>> ERROR [2018-03-05 13:15:32,882] ({pool-2-thread-2} Job.java[run]:184) -
>>>> Job failed
>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>> org.apache.thrift.transport.TTransportException:
>>>> java.net.ConnectException: Connection refused (Connection refused)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.
>>>> callRemoteFunction(RemoteInterpreterProcess.java:152)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_
>>>> create(RemoteInterpreter.java:154)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(
>>>> RemoteInterpreter.java:126)
>>>> at org.apache.zeppelin.interpreter.remote.
>>>> RemoteInterpreter.getFormType(RemoteInterpreter.java:265)
>>>> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:423)
>>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:182)
>>>> at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(
>>>> RemoteScheduler.java:307)
>>>> at java.util.concurrent.Executors$RunnableAdapter.
>>>> call(Executors.java:511)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$
>>>> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$
>>>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(
>>>> ThreadPoolExecutor.java:1149)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>>>> ThreadPoolExecutor.java:624)
>>>> at java.lang.Thread.run(Thread.java:748)
>>>> Caused by: org.apache.zeppelin.interpreter.InterpreterException:
>>>> org.apache.thrift.transport.TTransportException:
>>>> java.net.ConnectException: Connection refused (Connection refused)
>>>> at org.apache.zeppelin.interpreter.remote.ClientFactory.create(
>>>> ClientFactory.java:53)
>>>> at org.apache.zeppelin.interpreter.remote.ClientFactory.create(
>>>> ClientFactory.java:37)
>>>> at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(
>>>> BasePooledObjectFactory.java:60)
>>>> at org.apache.commons.pool2.impl.GenericObjectPool.create(
>>>> GenericObjectPool.java:861)
>>>> at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(
>>>> GenericObjectPool.java:435)
>>>> at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(
>>>> GenericObjectPool.java:363)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.
>>>> getClient(RemoteInterpreterProcess.java:73)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.
>>>> callRemoteFunction(RemoteInterpreterProcess.java:144)
>>>> ... 13 more
>>>> Caused by: org.apache.thrift.transport.TTransportException:
>>>> java.net.ConnectException: Connection refused (Connection refused)
>>>> at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
>>>> at org.apache.zeppelin.interpreter.remote.ClientFactory.create(
>>>> ClientFactory.java:51)
>>>> ... 20 more
>>>> Caused by: java.net.ConnectException: Connection refused (Connection
>>>> refused)
>>>> at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>> at java.net.AbstractPlainSocketImpl.doConnect(
>>>> AbstractPlainSocketImpl.java:350)
>>>> at java.net.AbstractPlainSocketImpl.connectToAddress(
>>>> AbstractPlainSocketImpl.java:206)
>>>> at java.net.AbstractPlainSocketImpl.connect(
>>>> AbstractPlainSocketImpl.java:188)
>>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>>>> at java.net.Socket.connect(Socket.java:589)
>>>> at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
>>>> ... 21 more
>>>> ERROR [2018-03-05 13:15:32,884] ({pool-2-thread-2}
>>>> NotebookServer.java[afterStatusChange]:2219)
>>>> - Error
>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>> org.apache.thrift.transport.TTransportException:
>>>> java.net.ConnectException: Connection refused (Connection refused)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.
>>>> callRemoteFunction(RemoteInterpreterProcess.java:152)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_
>>>> create(RemoteInterpreter.java:154)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(
>>>> RemoteInterpreter.java:126)
>>>> at org.apache.zeppelin.interpreter.remote.
>>>> RemoteInterpreter.getFormType(RemoteInterpreter.java:265)
>>>> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:423)
>>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:182)
>>>> at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(
>>>> RemoteScheduler.java:307)
>>>> at java.util.concurrent.Executors$RunnableAdapter.
>>>> call(Executors.java:511)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$
>>>> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$
>>>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(
>>>> ThreadPoolExecutor.java:1149)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>>>> ThreadPoolExecutor.java:624)
>>>> at java.lang.Thread.run(Thread.java:748)
>>>> Caused by: org.apache.zeppelin.interpreter.InterpreterException:
>>>> org.apache.thrift.transport.TTransportException:
>>>> java.net.ConnectException: Connection refused (Connection refused)
>>>> at org.apache.zeppelin.interpreter.remote.ClientFactory.create(
>>>> ClientFactory.java:53)
>>>> at org.apache.zeppelin.interpreter.remote.ClientFactory.create(
>>>> ClientFactory.java:37)
>>>> at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(
>>>> BasePooledObjectFactory.java:60)
>>>> at org.apache.commons.pool2.impl.GenericObjectPool.create(
>>>> GenericObjectPool.java:861)
>>>> at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(
>>>> GenericObjectPool.java:435)
>>>> at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(
>>>> GenericObjectPool.java:363)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.
>>>> getClient(RemoteInterpreterProcess.java:73)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.
>>>> callRemoteFunction(RemoteInterpreterProcess.java:144)
>>>> ... 13 more
>>>> Caused by: org.apache.thrift.transport.TTransportException:
>>>> java.net.ConnectException: Connection refused (Connection refused)
>>>> at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
>>>> at org.apache.zeppelin.interpreter.remote.ClientFactory.create(
>>>> ClientFactory.java:51)
>>>> ... 20 more
>>>> Caused by: java.net.ConnectException: Connection refused (Connection
>>>> refused)
>>>> at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>> at java.net.AbstractPlainSocketImpl.doConnect(
>>>> AbstractPlainSocketImpl.java:350)
>>>> at java.net.AbstractPlainSocketImpl.connectToAddress(
>>>> AbstractPlainSocketImpl.java:206)
>>>> at java.net.AbstractPlainSocketImpl.connect(
>>>> AbstractPlainSocketImpl.java:188)
>>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>>>> at java.net.Socket.connect(Socket.java:589)
>>>> at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
>>>> ... 21 more
>>>> WARN [2018-03-05 13:15:32,885] ({pool-2-thread-2}
>>>> NotebookServer.java[afterStatusChange]:2227)
>>>> - Job 20180302-135047_99102973 is finished, status: ERROR, exception:
>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>> org.apache.thrift.transport.TTransportException:
>>>> java.net.ConnectException: Connection refused (Connection refused), result:
>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>> org.apache.thrift.transport.TTransportException:
>>>> java.net.ConnectException: Connection refused (Connection refused)
>>>> INFO [2018-03-05 13:15:32,910] ({pool-2-thread-2}
>>>> VFSNotebookRepo.java[save]:221) - Saving note:2D6S6TB3U
>>>> INFO [2018-03-05 13:15:32,914] ({pool-2-thread-2}
>>>> SchedulerFactory.java[jobFinished]:115) - Job 20180302-135047_99102973
>>>> finished by scheduler org.apache.zeppelin.interpreter.remote.
>>>> RemoteInterpreter-shared_session
>>>>
>>>> 2018-03-02 15:07 GMT+09:00 小野圭二 <onoke...@gmail.com>:
>>>>
>>>>> Hi Jeff,
>>>>>
>>>>> No one graved 63039 so far on Mac.
>>>>> Yes, this issue has been reproduced as changing the port NO randomly.
>>>>>
>>>>> Ono
>>>>>
>>>>>
>>>>> 2018-03-02 14:49 GMT+09:00 Jeff Zhang <zjf...@gmail.com>:
>>>>>
>>>>>>
>>>>>> It is possible that 63039 is used by other processes, as Mac can not
>>>>>> detect whether this port is used or not from my experience.
>>>>>>
>>>>>> And can you reproduce this issue consistently ? Because each time
>>>>>> zeppelin will chose a random port as long as you don't specify the
>>>>>> portRange, so if you reproduce it consistently, it might due to other
>>>>>> issues.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> 小野圭二 <onoke...@gmail.com>于2018年3月2日周五 下午1:32写道：
>>>>>>
>>>>>>> Hi there,
>>>>>>>
>>>>>>> I am so confusing now, let me know why if You know this, please.
>>>>>>>
>>>>>>> The env:
>>>>>>> Mac: High Sierra ver.10.13.3
>>>>>>> Zep: 0.7.3
>>>>>>> Spark: 2.2.0
>>>>>>> Java: 1.8
>>>>>>> notebook script: %pyspark print 'aaa' (just it
>>>>>>> for test)
>>>>>>>
>>>>>>> In this env, i got the error in log as
>>>>>>>
>>>>>>> ERROR ({pool-2-thread-2} Job.java[run]:184) - Job failed
>>>>>>>
>>>>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>>>>> org.apache.thrift.transport.TTransportException:
>>>>>>> java.net.ConnectException: Connection refused (Connection refused)
>>>>>>>
>>>>>>>
>>>>>>> Indeed, before running notebook, RemoteInterpreterManagedProcess.java
>>>>>>> prepares a Port for callback to Thrift.
>>>>>>>
>>>>>>>
>>>>>>> INFO ({pool-2-thread-1} RemoteInterpreterManagedProcess.java[start]:130)
>>>>>>> - Thrift server for callback will start. Port: 63039
>>>>>>>
>>>>>>>
>>>>>>> I guess, the reason why the error was Zeppelin could not attache to
>>>>>>> the Thrift PORT.
>>>>>>>
>>>>>>>
>>>>>>> And my confusion is, when i run the same module on CentOS, it worked
>>>>>>> fine.
>>>>>>> CentOS: 7
>>>>>>> Zep: same
>>>>>>> Spark: same
>>>>>>> Java: same
>>>>>>> notebook script: same
>>>>>>>
>>>>>>> And there was no the Thift port description in the log file. From
>>>>>>> this consequence, the fine working was cause of did not use Thrift port
>>>>>>> on
>>>>>>> CentOS.
>>>>>>>
>>>>>>>
>>>>>>> Do You know actually there is different behavior between Mac and
>>>>>>> CentOS? And i am so appreciated if You know the workaround on Mac.
>>>>>>>
>>>>>>>
>>>>>>> Best,
>>>>>>> Ono
>>>>>>>
>>>>>>
>>>>>
>>>>
>>
