Re: [DISCUSS] Release 0.12.0 (1.0.0)
+1 for deprecating Java 8 and dropping Spark 3.2. I also have a working version of "unofficial" Spark 4.0 support I can submit a PR for. Unofficial because I don't want to mark 4.0.0 as supported before it's released, but I got the Spark interpreter tests to pass with 4.0.0-preview1. One thing I noticed while trying to build a release with Spark 4.0 support is that the 0.12.0-SNAPSHOT build I made broke in our deployment, because it looks like there was a change that forced keytab/principal settings to only be loaded from env vars or system props, and not the zeppelin-site.xml config. I ended up just backporting it to a 0.11.2 build and Spark 4.0.0-preview1 seems to be working fine. Adam On Wed, Sep 11, 2024 at 2:21 AM Cheng Pan wrote: > Thanks for bringing up this discussion, generally, I agree that we should > have a short release cycle. > > For the version number, I prefer to 0.12 instead of 1.0, because some > interpreters relies on external systems were not fully tested with Java 11, > I would like to give it more time to collect user feedbacks before enter > 1.0. > > +1 for marking Java 8 deprecated in the next release, meanwhile, if > someone is interested, we should also consider adding support for Java 17, > to prepare support for Spark 4.0. > > I would also like to propose dropping support for Spark 3.2, and making > Spark 3.5 as the default version. > > Currently, Zeppelin supports Spark 3.2 to 3.5. > > Spark 3.2 released at Oct 2021, was EOL at Apr 2023. > Spark 3.3 released at Jun 2022, was EOL at Dec 2023. > Spark 3.4 released at Apr 2023, will be EOL at Oct 2024. > Spark 3.5 released at Aug 2024, it’s a LTS version. > > For users who want to use lower Spark versions, Livy and Kyuubi(for now, > only supports Spark SQL via the JDBC interpreter) are alternatives. > > Thanks, > Cheng Pan > > > > > On Sep 11, 2024, at 10:24, Jongyoul Lee wrote: > > > > Hello, > > > > I would like to discuss a new release with the community. > > > > Here is what we are working on: > > - Docker enhancement > > - Java 8 deprecation > > - New UI > > - Raft deprecation > > > > If you have any questions or ideas for the next release please feel > > free to leave your comments. > > > > Focusing on contributing and reviewing relevant tasks, I hope we do > > the next release by the end of this month. > > > > -- > > Best regards, > > Jongyoul Lee > > -- Adam Binford
Re: [DISCUSS] Release 0.12.0 (1.0.0)
Adam, good to know that you are working on Spark 4 support, and Spark 4.0 preview2 is likely coming soon. It would be great if you can submit a draft PR even Spark 4.0 is not GA. > it looks like there was a change that forced keytab/principal settings to > only be loaded from env vars or system props, and not the zeppelin-site.xml this sounds like a regression that should be fixed, I’m not sure what’s the issue exactly being, if you are willing to debug, this project[1] can leverage docker compose to simplify setup a kerberized Hadoop environment. [1] https://github.com/awesome-kyuubi/hadoop-testing Thanks, Cheng Pan > On Sep 11, 2024, at 18:30, Adam Binford wrote: > > +1 for deprecating Java 8 and dropping Spark 3.2. > > I also have a working version of "unofficial" Spark 4.0 support I can > submit a PR for. Unofficial because I don't want to mark 4.0.0 as supported > before it's released, but I got the Spark interpreter tests to pass with > 4.0.0-preview1. > > One thing I noticed while trying to build a release with Spark 4.0 support > is that the 0.12.0-SNAPSHOT build I made broke in our deployment, because > it looks like there was a change that forced keytab/principal settings to > only be loaded from env vars or system props, and not the zeppelin-site.xml > config. I ended up just backporting it to a 0.11.2 build and Spark > 4.0.0-preview1 seems to be working fine. > > Adam > > On Wed, Sep 11, 2024 at 2:21 AM Cheng Pan wrote: > >> Thanks for bringing up this discussion, generally, I agree that we should >> have a short release cycle. >> >> For the version number, I prefer to 0.12 instead of 1.0, because some >> interpreters relies on external systems were not fully tested with Java 11, >> I would like to give it more time to collect user feedbacks before enter >> 1.0. >> >> +1 for marking Java 8 deprecated in the next release, meanwhile, if >> someone is interested, we should also consider adding support for Java 17, >> to prepare support for Spark 4.0. >> >> I would also like to propose dropping support for Spark 3.2, and making >> Spark 3.5 as the default version. >> >> Currently, Zeppelin supports Spark 3.2 to 3.5. >> >> Spark 3.2 released at Oct 2021, was EOL at Apr 2023. >> Spark 3.3 released at Jun 2022, was EOL at Dec 2023. >> Spark 3.4 released at Apr 2023, will be EOL at Oct 2024. >> Spark 3.5 released at Aug 2024, it’s a LTS version. >> >> For users who want to use lower Spark versions, Livy and Kyuubi(for now, >> only supports Spark SQL via the JDBC interpreter) are alternatives. >> >> Thanks, >> Cheng Pan >> >> >> >>> On Sep 11, 2024, at 10:24, Jongyoul Lee wrote: >>> >>> Hello, >>> >>> I would like to discuss a new release with the community. >>> >>> Here is what we are working on: >>> - Docker enhancement >>> - Java 8 deprecation >>> - New UI >>> - Raft deprecation >>> >>> If you have any questions or ideas for the next release please feel >>> free to leave your comments. >>> >>> Focusing on contributing and reviewing relevant tasks, I hope we do >>> the next release by the end of this month. >>> >>> -- >>> Best regards, >>> Jongyoul Lee >> >> > > -- > Adam Binford
Re: [DISCUSS] Release 0.12.0 (1.0.0)
> > this sounds like a regression that should be fixed, I’m not sure what’s > the issue exactly being This was specifically what I hit: https://github.com/apache/zeppelin/blob/940cf13935ec9ed989c7afeec98447c2e7eb5a7e/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/FileSystemStorage.java#L61 which was updated to use "getStaticString" in https://github.com/apache/zeppelin/commit/3eae255c109aa224c5b5ae3f2c986510d2a26ee9 I'll clean up and make the PR for Spark 4 stuff soon. On Wed, Sep 11, 2024 at 9:52 AM Cheng Pan wrote: > Adam, good to know that you are working on Spark 4 support, and Spark 4.0 > preview2 is likely coming soon. It would be great if you can submit a draft > PR even Spark 4.0 is not GA. > > > it looks like there was a change that forced keytab/principal settings to > > only be loaded from env vars or system props, and not the > zeppelin-site.xml > > this sounds like a regression that should be fixed, I’m not sure what’s > the issue exactly being, if you are willing to debug, this project[1] can > leverage docker compose to simplify setup a kerberized Hadoop environment. > > [1] https://github.com/awesome-kyuubi/hadoop-testing > > Thanks, > Cheng Pan > > > > > On Sep 11, 2024, at 18:30, Adam Binford wrote: > > > > +1 for deprecating Java 8 and dropping Spark 3.2. > > > > I also have a working version of "unofficial" Spark 4.0 support I can > > submit a PR for. Unofficial because I don't want to mark 4.0.0 as > supported > > before it's released, but I got the Spark interpreter tests to pass with > > 4.0.0-preview1. > > > > One thing I noticed while trying to build a release with Spark 4.0 > support > > is that the 0.12.0-SNAPSHOT build I made broke in our deployment, because > > it looks like there was a change that forced keytab/principal settings to > > only be loaded from env vars or system props, and not the > zeppelin-site.xml > > config. I ended up just backporting it to a 0.11.2 build and Spark > > 4.0.0-preview1 seems to be working fine. > > > > Adam > > > > On Wed, Sep 11, 2024 at 2:21 AM Cheng Pan wrote: > > > >> Thanks for bringing up this discussion, generally, I agree that we > should > >> have a short release cycle. > >> > >> For the version number, I prefer to 0.12 instead of 1.0, because some > >> interpreters relies on external systems were not fully tested with Java > 11, > >> I would like to give it more time to collect user feedbacks before enter > >> 1.0. > >> > >> +1 for marking Java 8 deprecated in the next release, meanwhile, if > >> someone is interested, we should also consider adding support for Java > 17, > >> to prepare support for Spark 4.0. > >> > >> I would also like to propose dropping support for Spark 3.2, and making > >> Spark 3.5 as the default version. > >> > >> Currently, Zeppelin supports Spark 3.2 to 3.5. > >> > >> Spark 3.2 released at Oct 2021, was EOL at Apr 2023. > >> Spark 3.3 released at Jun 2022, was EOL at Dec 2023. > >> Spark 3.4 released at Apr 2023, will be EOL at Oct 2024. > >> Spark 3.5 released at Aug 2024, it’s a LTS version. > >> > >> For users who want to use lower Spark versions, Livy and Kyuubi(for now, > >> only supports Spark SQL via the JDBC interpreter) are alternatives. > >> > >> Thanks, > >> Cheng Pan > >> > >> > >> > >>> On Sep 11, 2024, at 10:24, Jongyoul Lee wrote: > >>> > >>> Hello, > >>> > >>> I would like to discuss a new release with the community. > >>> > >>> Here is what we are working on: > >>> - Docker enhancement > >>> - Java 8 deprecation > >>> - New UI > >>> - Raft deprecation > >>> > >>> If you have any questions or ideas for the next release please feel > >>> free to leave your comments. > >>> > >>> Focusing on contributing and reviewing relevant tasks, I hope we do > >>> the next release by the end of this month. > >>> > >>> -- > >>> Best regards, > >>> Jongyoul Lee > >> > >> > > > > -- > > Adam Binford > > -- Adam Binford
[jira] [Created] (ZEPPELIN-6085) Add configuration option for setting the default UI
Chan Ho Lee created ZEPPELIN-6085: - Summary: Add configuration option for setting the default UI Key: ZEPPELIN-6085 URL: https://issues.apache.org/jira/browse/ZEPPELIN-6085 Project: Zeppelin Issue Type: Improvement Reporter: Chan Ho Lee Assignee: Chan Ho Lee This issue introduces a new configuration option, `zeppelin.default.ui`, that allows users to define which UI is served by default in Zeppelin. The available options are `new` and `classic`. By default, the value is set to `new`, which will serve the new UI at the root path(`"/"`) and the classic UI at `"/classic"`. If the configuration is set to `classic`, the classic UI will be served at `"/"` and the new UI at `"/"` The new UI has recently become the default, but it is not yet fully complete, which may cause inconvenience for users. By adding this option, we aim to reduce the inconvenience for users who are not ready to migrate to the new UI -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ZEPPELIN-6086) Support Spark 4.0
Adam Binford created ZEPPELIN-6086: -- Summary: Support Spark 4.0 Key: ZEPPELIN-6086 URL: https://issues.apache.org/jira/browse/ZEPPELIN-6086 Project: Zeppelin Issue Type: Improvement Components: spark Reporter: Adam Binford Now that Spark 4.0.0-preview1 is out (and soon to be preview2), we can start adding support for Spark 4+. Currently all attempts to use Spark 4 fail, even with version checking disabled, because there are no shims for 4+ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ZEPPELIN-6087) Create Oracle Database Interpreter
Pablo Silberkasten created ZEPPELIN-6087: Summary: Create Oracle Database Interpreter Key: ZEPPELIN-6087 URL: https://issues.apache.org/jira/browse/ZEPPELIN-6087 Project: Zeppelin Issue Type: New Feature Components: documentation, GUI, JdbcInterpreter, Kubernetes, zeppelin-site-repo Affects Versions: 0.12.0 Reporter: Pablo Silberkasten Add an Oracle Database Interpreter. Using the Oracle JDBC Driver, UCP and its specific best practices and configuration values. -- This message was sent by Atlassian Jira (v8.20.10#820010)