Hi,
We are evaluating Zeppelin and are excited about its potential. Right now,
we are using it to access Presto and Vertica. We downloaded
https://github.com/apache/incubator-zeppelin/pull/68 for the Presto
interpreter and are using PostgreSqlInterpreter to access Vertica.
The main issue we are facing is related to serial execution of queries.
Since the default scheduler is FIFOScheduler, when a Vertica query is
running in one notebook, all other Vertica queries issued from the same or
different notes are pending.
We changed the getScheduler method in PostgreSqlInterpreter to return
ParallelScheduler.
return SchedulerFactory.singleton().createOrGetParallelScheduler(
PostgreSqlInterpreter.class.getName() + this.hashCode(), 2);
But this resulted in the exception below. Is this expected behavior? Does
the PostgreSqlInterpreter have to be redesigned to support parallel
execution of queries?
In zeppelin-interpreter-psql-pwagle-MacBook-Pro.local.log:
INFO [2015-12-03 10:33:35,891] ({pool-1-thread-2}
RemoteInterpreterServer.java[createInterpreter]:168) - Instantiate
interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter
INFO [2015-12-03 10:33:35,940] ({pool-2-thread-2}
SchedulerFactory.java[jobStarted]:132) - Job
remoteInterpretJob_1449167615939 started by scheduler
org.apache.zeppelin.postgresql.PostgreSqlInterpreter761108485
INFO [2015-12-03 10:33:35,942] ({pool-2-thread-2}
PostgreSqlInterpreter.java[open]:132) - Open psql connection!
INFO [2015-12-03 10:33:35,942] ({pool-2-thread-2}
PostgreSqlInterpreter.java[close]:182) - Close psql connection!
INFO [2015-12-03 10:33:36,511] ({pool-2-thread-2}
SqlCompleter.java[getSqlKeywordsCompletions]:144) - JDBC
DriverName:/vertica.jar-sql.keywords
INFO [2015-12-03 10:33:37,779] ({pool-2-thread-3}
SchedulerFactory.java[jobStarted]:132) - Job
remoteInterpretJob_1449167617779 started by scheduler
org.apache.zeppelin.postgresql.PostgreSqlInterpreter761108485
INFO [2015-12-03 10:33:49,352] ({pool-2-thread-2}
PostgreSqlInterpreter.java[open]:153) - Successfully created psql connection
INFO [2015-12-03 10:33:49,352] ({pool-2-thread-2}
PostgreSqlInterpreter.java[interpret]:288) - Run SQL command '
select sleep(30), 1;'
INFO [2015-12-03 10:33:49,352] ({pool-2-thread-3}
PostgreSqlInterpreter.java[interpret]:288) - Run SQL command '
select sleep(30), 2;'
ERROR [2015-12-03 10:34:49,482] ({pool-2-thread-3}
PostgreSqlInterpreter.java[executeSql]:271) - Cannot run
select sleep(30), 2;
java.sql.SQLException: [Vertica][JDBC](12060) ResultSet object has been
closed.
at com.vertica.jdbc.SForwardResultSet.checkIfOpen(Unknown Source)
at com.vertica.jdbc.SForwardResultSet.next(Unknown Source)
at
org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:235)
at
org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:289)
at
org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:57)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:299)
at org.apache.zeppelin.scheduler.Job.run(Job.java:170)
at
org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:157)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
ERROR [2015-12-03 10:34:49,482] ({pool-2-thread-2} Job.java[run]:183) - Job
failed
org.apache.zeppelin.interpreter.InterpreterException:
java.lang.NullPointerException
at
org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:61)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:299)
at org.apache.zeppelin.scheduler.Job.run(Job.java:170)
at
org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:157)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.NullPointerException
at
org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:262)
at
org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:289)
at
org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:57)
... 11 more
INFO [2015-12-03 10:34:49,489] ({pool-2-thread-3}
SchedulerFactory.java[jobFinished]:138) - Job
remoteInterpretJob_1449167617779 finished by scheduler
org.apache.zeppelin.postgresql.PostgreSqlInterpreter761108485
INFO [2015-12-03 10:34:49,490] ({pool-2-thread-2}
SchedulerFactory.java[jobFinished]:138) - Job
remoteInterpretJob_1449167615939 finished by scheduler
org.apache.zeppelin.postgresql.PostgreSqlInterpreter761108485
Thanks,
Prasad
