Hello, Thank you for all your opinions and consideration. I appreciate it.
To summarize, - Drop Java 8 - Drop Spark 3.2, 3.3, 3.4 and make 3.5 as default - Drop outdated Flink - Support JDK 17 and Spark 4.x - Drop/replace outdated codes/libraries like Raft and docker-client - Introduce new UI - Prefer 0.12 :-) It should be good to have it all in the next release but we prioritize getting feedback for the new UI and I hope this is the major change for the next release. We can have a new release soon again. Best regards, JL 2024년 9월 12일 (목) 오후 10:33, Philipp Dallig <philipp.dal...@gmail.com>님이 작성: > > Hello, > > > Thank you for preparing a new version. We should get rid of Java 8 as soon as > possible. There are several libraries that cannot be updated because of Java > 8. I tend to prefer to switch directly to JDK17. This is supported by Spark > 3.5, for example. Unfortunately, Hadoop 3 does not work with JDK 17. > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Java+Versions > > Please keep in mind that Java 8 has a much longer support area than 11. > Most Linux distributions use openjdk and some people need to do the backport > process. RedHat as a contributor is dropping support for JDK 11 next month. > https://endoflife.date/redhat-build-of-openjdk > Other companies like Microsoft seem to support OpenJDK 11 longer. > https://endoflife.date/microsoft-build-of-openjdk > > I have seen the pull request with Docker examples. But I don't know if we > should focus on Zeppelin in Docker. The current implementation should also be > improved. The following start command doesn't look so good ... > https://github.com/apache/zeppelin/blob/940cf13935ec9ed989c7afeec98447c2e7eb5a7e/zeppelin-plugins/launcher/docker/src/main/java/org/apache/zeppelin/interpreter/launcher/DockerInterpreterProcess.java#L194-L201 > The development of our used docker-client library `com.spotify:docker-client` > was stopped a while ago. https://github.com/spotify/docker-client > Spotify has probably switched from its own product Helios > (https://github.com/spotify/helios) to Kubernetes. I have been running > Zeppelin in Kubernetes for a long time. Getting started with Docker is of > course much easier. > > The raft implementation library is also no longer being developed and we > should remove it. https://github.com/atomix/atomix-archive > This will also eliminate a lot of complexity. Maybe we can keep interfaces so > that another implementation is possible via plugin, for example. > > The EOL versions of Spark and Flink should be removed to speed up CI and > reduce complexity. > > As written on GitHub, I like the switch to the new user interface. I hope > that development will be easier with a newer Angular stack. > > Due to the many changes, I prefer a 0.12 version. > > Best Regards > > Philipp Dallig > > Am 11.09.24 um 12:30 schrieb Adam Binford: > > +1 for deprecating Java 8 and dropping Spark 3.2. > > I also have a working version of "unofficial" Spark 4.0 support I can submit > a PR for. Unofficial because I don't want to mark 4.0.0 as supported before > it's released, but I got the Spark interpreter tests to pass with > 4.0.0-preview1. > > One thing I noticed while trying to build a release with Spark 4.0 support is > that the 0.12.0-SNAPSHOT build I made broke in our deployment, because it > looks like there was a change that forced keytab/principal settings to only > be loaded from env vars or system props, and not the zeppelin-site.xml > config. I ended up just backporting it to a 0.11.2 build and Spark > 4.0.0-preview1 seems to be working fine. > > Adam > > On Wed, Sep 11, 2024 at 2:21 AM Cheng Pan <pan3...@gmail.com> wrote: >> >> Thanks for bringing up this discussion, generally, I agree that we should >> have a short release cycle. >> >> For the version number, I prefer to 0.12 instead of 1.0, because some >> interpreters relies on external systems were not fully tested with Java 11, >> I would like to give it more time to collect user feedbacks before enter 1.0. >> >> +1 for marking Java 8 deprecated in the next release, meanwhile, if someone >> is interested, we should also consider adding support for Java 17, to >> prepare support for Spark 4.0. >> >> I would also like to propose dropping support for Spark 3.2, and making >> Spark 3.5 as the default version. >> >> Currently, Zeppelin supports Spark 3.2 to 3.5. >> >> Spark 3.2 released at Oct 2021, was EOL at Apr 2023. >> Spark 3.3 released at Jun 2022, was EOL at Dec 2023. >> Spark 3.4 released at Apr 2023, will be EOL at Oct 2024. >> Spark 3.5 released at Aug 2024, it’s a LTS version. >> >> For users who want to use lower Spark versions, Livy and Kyuubi(for now, >> only supports Spark SQL via the JDBC interpreter) are alternatives. >> >> Thanks, >> Cheng Pan >> >> >> >> > On Sep 11, 2024, at 10:24, Jongyoul Lee <jongy...@gmail.com> wrote: >> > >> > Hello, >> > >> > I would like to discuss a new release with the community. >> > >> > Here is what we are working on: >> > - Docker enhancement >> > - Java 8 deprecation >> > - New UI >> > - Raft deprecation >> > >> > If you have any questions or ideas for the next release please feel >> > free to leave your comments. >> > >> > Focusing on contributing and reviewing relevant tasks, I hope we do >> > the next release by the end of this month. >> > >> > -- >> > Best regards, >> > Jongyoul Lee >> > > > -- > Adam Binford -- Best regards, Jongyoul Lee
