Arguments are really convincing; new Dataset API as well as performance
improvements is exiting, so I'm personally +1 on moving onto Java8.
However, I'm afraid Tencent is one of "the organizations stuck with Java7"
-- our IT Infra division wouldn't upgrade to Java7 until Java8 is out, and
wouldn't upgrade to Java8 until Java9 is out.
So:
(non-binding) +1 on dropping scala 2.10 support
(non-binding) -1 on dropping Java 7 support
* as long as we figure out a practical way to run
Spark with
JDK8 on JDK7 clusters, this -1 would then
definitely be +1
Thanks !
On Fri, Mar 25, 2016 at 10:28 AM, Koert Kuipers <ko...@tresata.com> wrote:
> i think that logic is reasonable, but then the same should also apply to
> scala 2.10, which is also unmaintained/unsupported at this point (basically
> has been since march 2015 except for one hotfix due to a license
> incompatibility)
>
> who wants to support scala 2.10 three years after they did the last
> maintenance release?
>
>
> On Thu, Mar 24, 2016 at 9:59 PM, Mridul Muralidharan <mri...@gmail.com>
> wrote:
>
>> Removing compatibility (with jdk, etc) can be done with a major release-
>> given that 7 has been EOLed a while back and is now unsupported, we have to
>> decide if we drop support for it in 2.0 or 3.0 (2+ years from now).
>>
>> Given the functionality & performance benefits of going to jdk8, future
>> enhancements relevant in 2.x timeframe ( scala, dependencies) which
>> requires it, and simplicity wrt code, test & support it looks like a good
>> checkpoint to drop jdk7 support.
>>
>> As already mentioned in the thread, existing yarn clusters are unaffected
>> if they want to continue running jdk7 and yet use spark2 (install jdk8 on
>> all nodes and use it via JAVA_HOME, or worst case distribute jdk8 as
>> archive - suboptimal).
>> I am unsure about mesos (standalone might be easier upgrade I guess ?).
>>
>>
>> Proposal is for 1.6x line to continue to be supported with critical
>> fixes; newer features will require 2.x and so jdk8
>>
>> Regards
>> Mridul
>>
>>
>> On Thursday, March 24, 2016, Marcelo Vanzin <van...@cloudera.com> wrote:
>>
>>> On Thu, Mar 24, 2016 at 4:50 PM, Reynold Xin <r...@databricks.com>
>>> wrote:
>>> > If you want to go down that route, you should also ask somebody who
>>> has had
>>> > experience managing a large organization's applications and try to
>>> update
>>> > Scala version.
>>>
>>> I understand both sides. But if you look at what I've been asking
>>> since the beginning, it's all about the cost and benefits of dropping
>>> support for java 1.7.
>>>
>>> The biggest argument in your original e-mail is about testing. And the
>>> testing cost is much bigger for supporting scala 2.10 than it is for
>>> supporting java 1.7. If you read one of my earlier replies, it should
>>> be even possible to just do everything in a single job - compile for
>>> java 7 and still be able to test things in 1.8, including lambdas,
>>> which seems to be the main thing you were worried about.
>>>
>>>
>>> > On Thu, Mar 24, 2016 at 4:48 PM, Marcelo Vanzin <van...@cloudera.com>
>>> wrote:
>>> >>
>>> >> On Thu, Mar 24, 2016 at 4:46 PM, Reynold Xin <r...@databricks.com>
>>> wrote:
>>> >> > Actually it's *way* harder to upgrade Scala from 2.10 to 2.11, than
>>> >> > upgrading the JVM runtime from 7 to 8, because Scala 2.10 and 2.11
>>> are
>>> >> > not
>>> >> > binary compatible, whereas JVM 7 and 8 are binary compatible except
>>> >> > certain
>>> >> > esoteric cases.
>>> >>
>>> >> True, but ask anyone who manages a large cluster how long it would
>>> >> take them to upgrade the jdk across their cluster and validate all
>>> >> their applications and everything... binary compatibility is a tiny
>>> >> drop in that bucket.
>>> >>
>>> >> --
>>> >> Marcelo
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Marcelo
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
>>> For additional commands, e-mail: dev-h...@spark.apache.org
>>>
>>>
>
