Re: [DISCUSS] Release 0.12.0 (1.0.0)

2024-09-11 Thread Adam Binford
+1 for deprecating Java 8 and dropping Spark 3.2.

I also have a working version of "unofficial" Spark 4.0 support I can
submit a PR for. Unofficial because I don't want to mark 4.0.0 as supported
before it's released, but I got the Spark interpreter tests to pass with
4.0.0-preview1.

One thing I noticed while trying to build a release with Spark 4.0 support
is that the 0.12.0-SNAPSHOT build I made broke in our deployment, because
it looks like there was a change that forced keytab/principal settings to
only be loaded from env vars or system props, and not the zeppelin-site.xml
config. I ended up just backporting it to a 0.11.2 build and Spark
4.0.0-preview1 seems to be working fine.

Adam

On Wed, Sep 11, 2024 at 2:21 AM Cheng Pan  wrote:

> Thanks for bringing up this discussion, generally, I agree that we should
> have a short release cycle.
>
> For the version number, I prefer to 0.12 instead of 1.0, because some
> interpreters relies on external systems were not fully tested with Java 11,
> I would like to give it more time to collect user feedbacks before enter
> 1.0.
>
> +1 for marking Java 8 deprecated in the next release, meanwhile, if
> someone is interested, we should also consider adding support for Java 17,
> to prepare support for Spark 4.0.
>
> I would also like to propose dropping support for Spark 3.2, and making
> Spark 3.5 as the default version.
>
> Currently, Zeppelin supports Spark 3.2 to 3.5.
>
> Spark 3.2 released at Oct 2021, was EOL at Apr 2023.
> Spark 3.3 released at Jun 2022, was EOL at Dec 2023.
> Spark 3.4 released at Apr 2023, will be EOL at Oct 2024.
> Spark 3.5 released at Aug 2024, it’s a LTS version.
>
> For users who want to use lower Spark versions, Livy and Kyuubi(for now,
> only supports Spark SQL via the JDBC interpreter) are alternatives.
>
> Thanks,
> Cheng Pan
>
>
>
> > On Sep 11, 2024, at 10:24, Jongyoul Lee  wrote:
> >
> > Hello,
> >
> > I would like to discuss a new release with the community.
> >
> > Here is what we are working on:
> > - Docker enhancement
> > - Java 8 deprecation
> > - New UI
> > - Raft deprecation
> >
> > If you have any questions or ideas for the next release please feel
> > free to leave your comments.
> >
> > Focusing on contributing and reviewing relevant tasks, I hope we do
> > the next release by the end of this month.
> >
> > --
> > Best regards,
> > Jongyoul Lee
>
>

-- 
Adam Binford


Re: [DISCUSS] Release 0.12.0 (1.0.0)

2024-09-11 Thread Cheng Pan
Adam, good to know that you are working on Spark 4 support, and Spark 4.0 
preview2 is likely coming soon. It would be great if you can submit a draft PR 
even Spark 4.0 is not GA. 

> it looks like there was a change that forced keytab/principal settings to
> only be loaded from env vars or system props, and not the zeppelin-site.xml

this sounds like a regression that should be fixed, I’m not sure what’s the 
issue exactly being, if you are willing to debug, this project[1] can leverage 
docker compose to simplify setup a kerberized Hadoop environment.

[1] https://github.com/awesome-kyuubi/hadoop-testing

Thanks,
Cheng Pan



> On Sep 11, 2024, at 18:30, Adam Binford  wrote:
> 
> +1 for deprecating Java 8 and dropping Spark 3.2.
> 
> I also have a working version of "unofficial" Spark 4.0 support I can
> submit a PR for. Unofficial because I don't want to mark 4.0.0 as supported
> before it's released, but I got the Spark interpreter tests to pass with
> 4.0.0-preview1.
> 
> One thing I noticed while trying to build a release with Spark 4.0 support
> is that the 0.12.0-SNAPSHOT build I made broke in our deployment, because
> it looks like there was a change that forced keytab/principal settings to
> only be loaded from env vars or system props, and not the zeppelin-site.xml
> config. I ended up just backporting it to a 0.11.2 build and Spark
> 4.0.0-preview1 seems to be working fine.
> 
> Adam
> 
> On Wed, Sep 11, 2024 at 2:21 AM Cheng Pan  wrote:
> 
>> Thanks for bringing up this discussion, generally, I agree that we should
>> have a short release cycle.
>> 
>> For the version number, I prefer to 0.12 instead of 1.0, because some
>> interpreters relies on external systems were not fully tested with Java 11,
>> I would like to give it more time to collect user feedbacks before enter
>> 1.0.
>> 
>> +1 for marking Java 8 deprecated in the next release, meanwhile, if
>> someone is interested, we should also consider adding support for Java 17,
>> to prepare support for Spark 4.0.
>> 
>> I would also like to propose dropping support for Spark 3.2, and making
>> Spark 3.5 as the default version.
>> 
>> Currently, Zeppelin supports Spark 3.2 to 3.5.
>> 
>> Spark 3.2 released at Oct 2021, was EOL at Apr 2023.
>> Spark 3.3 released at Jun 2022, was EOL at Dec 2023.
>> Spark 3.4 released at Apr 2023, will be EOL at Oct 2024.
>> Spark 3.5 released at Aug 2024, it’s a LTS version.
>> 
>> For users who want to use lower Spark versions, Livy and Kyuubi(for now,
>> only supports Spark SQL via the JDBC interpreter) are alternatives.
>> 
>> Thanks,
>> Cheng Pan
>> 
>> 
>> 
>>> On Sep 11, 2024, at 10:24, Jongyoul Lee  wrote:
>>> 
>>> Hello,
>>> 
>>> I would like to discuss a new release with the community.
>>> 
>>> Here is what we are working on:
>>> - Docker enhancement
>>> - Java 8 deprecation
>>> - New UI
>>> - Raft deprecation
>>> 
>>> If you have any questions or ideas for the next release please feel
>>> free to leave your comments.
>>> 
>>> Focusing on contributing and reviewing relevant tasks, I hope we do
>>> the next release by the end of this month.
>>> 
>>> --
>>> Best regards,
>>> Jongyoul Lee
>> 
>> 
> 
> -- 
> Adam Binford



Re: [DISCUSS] Release 0.12.0 (1.0.0)

2024-09-11 Thread Adam Binford
>
> this sounds like a regression that should be fixed, I’m not sure what’s
> the issue exactly being


This was specifically what I hit:
https://github.com/apache/zeppelin/blob/940cf13935ec9ed989c7afeec98447c2e7eb5a7e/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/FileSystemStorage.java#L61

which was updated to use "getStaticString" in
https://github.com/apache/zeppelin/commit/3eae255c109aa224c5b5ae3f2c986510d2a26ee9

I'll clean up and make the PR for Spark 4 stuff soon.

On Wed, Sep 11, 2024 at 9:52 AM Cheng Pan  wrote:

> Adam, good to know that you are working on Spark 4 support, and Spark 4.0
> preview2 is likely coming soon. It would be great if you can submit a draft
> PR even Spark 4.0 is not GA.
>
> > it looks like there was a change that forced keytab/principal settings to
> > only be loaded from env vars or system props, and not the
> zeppelin-site.xml
>
> this sounds like a regression that should be fixed, I’m not sure what’s
> the issue exactly being, if you are willing to debug, this project[1] can
> leverage docker compose to simplify setup a kerberized Hadoop environment.
>
> [1] https://github.com/awesome-kyuubi/hadoop-testing
>
> Thanks,
> Cheng Pan
>
>
>
> > On Sep 11, 2024, at 18:30, Adam Binford  wrote:
> >
> > +1 for deprecating Java 8 and dropping Spark 3.2.
> >
> > I also have a working version of "unofficial" Spark 4.0 support I can
> > submit a PR for. Unofficial because I don't want to mark 4.0.0 as
> supported
> > before it's released, but I got the Spark interpreter tests to pass with
> > 4.0.0-preview1.
> >
> > One thing I noticed while trying to build a release with Spark 4.0
> support
> > is that the 0.12.0-SNAPSHOT build I made broke in our deployment, because
> > it looks like there was a change that forced keytab/principal settings to
> > only be loaded from env vars or system props, and not the
> zeppelin-site.xml
> > config. I ended up just backporting it to a 0.11.2 build and Spark
> > 4.0.0-preview1 seems to be working fine.
> >
> > Adam
> >
> > On Wed, Sep 11, 2024 at 2:21 AM Cheng Pan  wrote:
> >
> >> Thanks for bringing up this discussion, generally, I agree that we
> should
> >> have a short release cycle.
> >>
> >> For the version number, I prefer to 0.12 instead of 1.0, because some
> >> interpreters relies on external systems were not fully tested with Java
> 11,
> >> I would like to give it more time to collect user feedbacks before enter
> >> 1.0.
> >>
> >> +1 for marking Java 8 deprecated in the next release, meanwhile, if
> >> someone is interested, we should also consider adding support for Java
> 17,
> >> to prepare support for Spark 4.0.
> >>
> >> I would also like to propose dropping support for Spark 3.2, and making
> >> Spark 3.5 as the default version.
> >>
> >> Currently, Zeppelin supports Spark 3.2 to 3.5.
> >>
> >> Spark 3.2 released at Oct 2021, was EOL at Apr 2023.
> >> Spark 3.3 released at Jun 2022, was EOL at Dec 2023.
> >> Spark 3.4 released at Apr 2023, will be EOL at Oct 2024.
> >> Spark 3.5 released at Aug 2024, it’s a LTS version.
> >>
> >> For users who want to use lower Spark versions, Livy and Kyuubi(for now,
> >> only supports Spark SQL via the JDBC interpreter) are alternatives.
> >>
> >> Thanks,
> >> Cheng Pan
> >>
> >>
> >>
> >>> On Sep 11, 2024, at 10:24, Jongyoul Lee  wrote:
> >>>
> >>> Hello,
> >>>
> >>> I would like to discuss a new release with the community.
> >>>
> >>> Here is what we are working on:
> >>> - Docker enhancement
> >>> - Java 8 deprecation
> >>> - New UI
> >>> - Raft deprecation
> >>>
> >>> If you have any questions or ideas for the next release please feel
> >>> free to leave your comments.
> >>>
> >>> Focusing on contributing and reviewing relevant tasks, I hope we do
> >>> the next release by the end of this month.
> >>>
> >>> --
> >>> Best regards,
> >>> Jongyoul Lee
> >>
> >>
> >
> > --
> > Adam Binford
>
>

-- 
Adam Binford


[jira] [Created] (ZEPPELIN-6085) Add configuration option for setting the default UI

2024-09-11 Thread Chan Ho Lee (Jira)
Chan Ho Lee created ZEPPELIN-6085:
-

 Summary: Add configuration option for setting the default UI
 Key: ZEPPELIN-6085
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-6085
 Project: Zeppelin
  Issue Type: Improvement
Reporter: Chan Ho Lee
Assignee: Chan Ho Lee


This issue introduces a new configuration option, `zeppelin.default.ui`, that 
allows users to define which UI is served by default in Zeppelin.

The available options are `new` and `classic`.

By default, the value is set to `new`, which will serve the new UI at the root 
path(`"/"`) and the classic UI at `"/classic"`.

If the configuration is set to `classic`, the classic UI will be served at 
`"/"` and the new UI at `"/"`

The new UI has recently become the default, but it is not yet fully complete, 
which may cause inconvenience for users.

By adding this option, we aim to reduce the inconvenience for users who are not 
ready to migrate to the new UI



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (ZEPPELIN-6086) Support Spark 4.0

2024-09-11 Thread Adam Binford (Jira)
Adam Binford created ZEPPELIN-6086:
--

 Summary: Support Spark 4.0
 Key: ZEPPELIN-6086
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-6086
 Project: Zeppelin
  Issue Type: Improvement
  Components: spark
Reporter: Adam Binford


Now that Spark 4.0.0-preview1 is out (and soon to be preview2), we can start 
adding support for Spark 4+. Currently all attempts to use Spark 4 fail, even 
with version checking disabled, because there are no shims for 4+



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (ZEPPELIN-6087) Create Oracle Database Interpreter

2024-09-11 Thread Pablo Silberkasten (Jira)
Pablo Silberkasten created ZEPPELIN-6087:


 Summary: Create Oracle Database Interpreter
 Key: ZEPPELIN-6087
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-6087
 Project: Zeppelin
  Issue Type: New Feature
  Components: documentation, GUI, JdbcInterpreter, Kubernetes, 
zeppelin-site-repo
Affects Versions: 0.12.0
Reporter: Pablo Silberkasten


Add an Oracle Database Interpreter. Using the Oracle JDBC Driver, UCP and its 
specific best practices and configuration values.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)