GitHub user HyukjinKwon opened a pull request:
https://github.com/apache/zeppelin/pull/3206
[WIP][ZEPPELIN-3810] Support Spark 2.4
### What is this PR for?
Spark 2.4 changed it's Scala version from 2.11.8 to 2.11.12 (see
SPARK-24418).
There are two problems for this upgrade at Zeppelin side:
1.. Some methods that are used in private by reflection, for instance,
`loopPostInit` became inaccessible.
See:
-
https://github.com/scala/scala/blob/v2.11.8/src/repl/scala/tools/nsc/interpreter/ILoop.scala
-
https://github.com/scala/scala/blob/v2.11.12/src/repl/scala/tools/nsc/interpreter/ILoop.scala
To work around this, I manually ported `loopPostInit` at 2.11.8 to retain
the behaviour. Some functions that are commonly existing at both Scala 2.11.8
and Scala 2.11.12 are used inside of the new `loopPostInit` by reflection.
2.. Upgrade from 2.11.8 to 2.11.12 requires `jline.version` upgrade.
Otherwise, we will hit:
```
Caused by: java.lang.NoSuchMethodError:
jline.console.completer.CandidateListCompletionHandler.setPrintSpaceAfterFullCompletion(Z)V
at
scala.tools.nsc.interpreter.jline.JLineConsoleReader.initCompletion(JLineReader.scala:139)
```
To work around this, I tweaked this by upgrading jline from `2.12.1` to
`2.14.3`.
### What type of PR is it?
[Improvement]
### Todos
* [ ] - Wait until Spark 2.4.0 is officially released.
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3810
### How should this be tested?
Verified manually against Spark 2.4.0 RC3
### Questions:
* Does the licenses files need update? Yes
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HyukjinKwon/zeppelin ZEPPELIN-3810
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/3206.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3206
----
commit e2d224aadf0e54533837a9a89f8e9d3586aee3a9
Author: hyukjinkwon <gurwls223@...>
Date: 2018-10-17T14:41:29Z
Support Spark 2.4
----
---
