Hi, I'm deploying zeppelin service on k8s following this article. https://zeppelin.apache.org/docs/0.10.1/quickstart/kubernetes.html
I downloaded whole flink 1.16.0 binary distribution and some other related jar files and put them into my custom zeppelin-server image. # my custom zeppelin Dockerfile FROM apache/zeppelin:0.10.1 COPY ./flink-1.16.0 /opt/flink-1.16.0 COPY ./flink-1.16.0/opt/flink-python-1.16.0.jar /opt/flink-1.16.0/lib/flink-python-1.16.0.jar COPY ./flink-shaded-hadoop-2-uber-2.8.3-10.0.jar /opt/flink-1.16.0/lib/flink-shaded-hadoop-2-uber-2.8.3-10.0.jar COPY ./flink-sql-connector-hive-3.1.2_2.12-1.16.0.jar /opt/flink-1.16.0/lib/flink-sql-connector-hive-3.1.2_2.12-1.16.0.jar COPY ./flink-sql-connector-kafka-1.16.0.jar /opt/flink-1.16.0/lib/flink-sql-connector-kafka-1.16.0.jar COPY ./flink-sql-parquet-1.16.0.jar /opt/flink-1.16.0/lib/flink-sql-parquet-1.16.0.jar COPY ./hive-site.xml /etc/hive/hive-site.xml I edited yaml file by replacing zeppelin-server image with my own custom zeppelin image. After applying the yaml, I got zepplin started running. This is the yaml file I used to create related resources on k8s. https://github.com/apache/zeppelin/blob/v0.10.1/k8s/zeppelin-server.yaml Then I configured flink interpreter by setting FLINK_HOME=/opt/flink-1.16.0, which I had already put into image, and did some other configurations. But when I try to run the Batch Wordcount script in the notebook named "1. Flink Basics" shipped with zeppelin, zeppelin throw an exception shown as below. java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at java.util.ArrayList.rangeCheck(ArrayList.java:659) at java.util.ArrayList.get(ArrayList.java:435) at org.apache.zeppelin.interpreter.launcher.FlinkInterpreterLauncher.chooseFlinkAppJar(FlinkInterpreterLauncher.java:148) at org.apache.zeppelin.interpreter.launcher.FlinkInterpreterLauncher.buildEnvFromProperties(FlinkInterpreterLauncher.java:84) at org.apache.zeppelin.interpreter.launcher.StandardInterpreterLauncher.launchDirectly(StandardInterpreterLauncher.java:77) at org.apache.zeppelin.interpreter.launcher.InterpreterLauncher.launch(InterpreterLauncher.java:110) at org.apache.zeppelin.interpreter.InterpreterSetting.createInterpreterProcess(InterpreterSetting.java:856) at org.apache.zeppelin.interpreter.ManagedInterpreterGroup.getOrCreateInterpreterProcess(ManagedInterpreterGroup.java:66) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getOrCreateInterpreterProcess(RemoteInterpreter.java:104) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:154) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:126) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:271) at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:438) at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:69) at org.apache.zeppelin.scheduler.Job.run(Job.java:172) at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:132) at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:182) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) I followed the stacktrace and checked following code. It seems that zeppelin tries to find a jar in flink's lib folder with name containing substring "flink-dist_" but cannot find any jar file matching the name convention. https://github.com/apache/zeppelin/blob/v0.10.1/zeppelin-plugins/launcher/flink/src/main/java/org/apache/zeppelin/interpreter/launcher/FlinkInterpreterLauncher.java#L138 I checked flink distribution with version starting from 1.14 to 1.16. It looks like they changed the flink-dist jar file's naming convention from "flink-dist_${scala.binary.version}-${project.version}.jar" to "flink-dist-${project.version}.jar". Scala version part is gone, so zeppelin cannot find any flink dist jar with substring "flink-dist_" and throws exception shown above. So i assume the code should be modified like below. List<File> flinkDistFiles = Arrays.stream(flinkLibFolder.listFiles(file -> file.getName().contains("flink-dist"))) .collect(Collectors.toList()); I dont have ASF account so I can't post issue on jira. Please accept my apology if this is misusage of the user maillist. -- 祝好！
