> > this sounds like a regression that should be fixed, I’m not sure what’s > the issue exactly being
This was specifically what I hit: https://github.com/apache/zeppelin/blob/940cf13935ec9ed989c7afeec98447c2e7eb5a7e/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/FileSystemStorage.java#L61 which was updated to use "getStaticString" in https://github.com/apache/zeppelin/commit/3eae255c109aa224c5b5ae3f2c986510d2a26ee9 I'll clean up and make the PR for Spark 4 stuff soon. On Wed, Sep 11, 2024 at 9:52 AM Cheng Pan <pan3...@gmail.com> wrote: > Adam, good to know that you are working on Spark 4 support, and Spark 4.0 > preview2 is likely coming soon. It would be great if you can submit a draft > PR even Spark 4.0 is not GA. > > > it looks like there was a change that forced keytab/principal settings to > > only be loaded from env vars or system props, and not the > zeppelin-site.xml > > this sounds like a regression that should be fixed, I’m not sure what’s > the issue exactly being, if you are willing to debug, this project[1] can > leverage docker compose to simplify setup a kerberized Hadoop environment. > > [1] https://github.com/awesome-kyuubi/hadoop-testing > > Thanks, > Cheng Pan > > > > > On Sep 11, 2024, at 18:30, Adam Binford <adam...@gmail.com> wrote: > > > > +1 for deprecating Java 8 and dropping Spark 3.2. > > > > I also have a working version of "unofficial" Spark 4.0 support I can > > submit a PR for. Unofficial because I don't want to mark 4.0.0 as > supported > > before it's released, but I got the Spark interpreter tests to pass with > > 4.0.0-preview1. > > > > One thing I noticed while trying to build a release with Spark 4.0 > support > > is that the 0.12.0-SNAPSHOT build I made broke in our deployment, because > > it looks like there was a change that forced keytab/principal settings to > > only be loaded from env vars or system props, and not the > zeppelin-site.xml > > config. I ended up just backporting it to a 0.11.2 build and Spark > > 4.0.0-preview1 seems to be working fine. > > > > Adam > > > > On Wed, Sep 11, 2024 at 2:21 AM Cheng Pan <pan3...@gmail.com> wrote: > > > >> Thanks for bringing up this discussion, generally, I agree that we > should > >> have a short release cycle. > >> > >> For the version number, I prefer to 0.12 instead of 1.0, because some > >> interpreters relies on external systems were not fully tested with Java > 11, > >> I would like to give it more time to collect user feedbacks before enter > >> 1.0. > >> > >> +1 for marking Java 8 deprecated in the next release, meanwhile, if > >> someone is interested, we should also consider adding support for Java > 17, > >> to prepare support for Spark 4.0. > >> > >> I would also like to propose dropping support for Spark 3.2, and making > >> Spark 3.5 as the default version. > >> > >> Currently, Zeppelin supports Spark 3.2 to 3.5. > >> > >> Spark 3.2 released at Oct 2021, was EOL at Apr 2023. > >> Spark 3.3 released at Jun 2022, was EOL at Dec 2023. > >> Spark 3.4 released at Apr 2023, will be EOL at Oct 2024. > >> Spark 3.5 released at Aug 2024, it’s a LTS version. > >> > >> For users who want to use lower Spark versions, Livy and Kyuubi(for now, > >> only supports Spark SQL via the JDBC interpreter) are alternatives. > >> > >> Thanks, > >> Cheng Pan > >> > >> > >> > >>> On Sep 11, 2024, at 10:24, Jongyoul Lee <jongy...@gmail.com> wrote: > >>> > >>> Hello, > >>> > >>> I would like to discuss a new release with the community. > >>> > >>> Here is what we are working on: > >>> - Docker enhancement > >>> - Java 8 deprecation > >>> - New UI > >>> - Raft deprecation > >>> > >>> If you have any questions or ideas for the next release please feel > >>> free to leave your comments. > >>> > >>> Focusing on contributing and reviewing relevant tasks, I hope we do > >>> the next release by the end of this month. > >>> > >>> -- > >>> Best regards, > >>> Jongyoul Lee > >> > >> > > > > -- > > Adam Binford > > -- Adam Binford
