Many open source projects are aggressive, such as Oracle JDK and Ubuntu, But they provide stable commercial supporting.
In other words, the enterprises doesn't drop JDK7, might aslo do not drop Spark 1.x to adopt Spark 2.x early version. On Sun, Apr 3, 2016 at 10:29 PM -0700, "Reynold Xin" <r...@databricks.com> wrote: Since my original email, I've talked to a lot more users and looked at what various environments support. It is true that a lot of enterprises, and even some technology companies, are still using Java 7. One thing is that up until this date, users still can't install openjdk 8 on Ubuntu by default. I see that as an indication that it is too early to drop Java 7. Looking at the timeline, JDK release a major new version roughly every 3 years. We dropped Java 6 support one year ago, so from a timeline point of view we would be very aggressive here if we were to drop Java 7 support in Spark 2.0. Note that not dropping Java 7 support now doesn't mean we have to support Java 7 throughout Spark 2.x. We dropped Java 6 support in Spark 1.5, even though Spark 1.0 started with Java 6. In terms of testing, Josh has actually improved our test infra so now we would run the Java 8 tests: https://github.com/apache/spark/pull/12073 On Thu, Mar 24, 2016 at 8:51 PM, Liwei Lin <lwl...@gmail.com> wrote: Arguments are really convincing; new Dataset API as well as performance improvements is exiting, so I'm personally +1 on moving onto Java8. However, I'm afraid Tencent is one of "the organizations stuck with Java7" -- our IT Infra division wouldn't upgrade to Java7 until Java8 is out, and wouldn't upgrade to Java8 until Java9 is out. So: (non-binding) +1 on dropping scala 2.10 support (non-binding) -1 on dropping Java 7 support * as long as we figure out a practical way to run Spark with JDK8 on JDK7 clusters, this -1 would then definitely be +1 Thanks ! On Fri, Mar 25, 2016 at 10:28 AM, Koert Kuipers <ko...@tresata.com> wrote: i think that logic is reasonable, but then the same should also apply to scala 2.10, which is also unmaintained/unsupported at this point (basically has been since march 2015 except for one hotfix due to a license incompatibility) who wants to support scala 2.10 three years after they did the last maintenance release? On Thu, Mar 24, 2016 at 9:59 PM, Mridul Muralidharan <mri...@gmail.com> wrote: Removing compatibility (with jdk, etc) can be done with a major release- given that 7 has been EOLed a while back and is now unsupported, we have to decide if we drop support for it in 2.0 or 3.0 (2+ years from now). Given the functionality & performance benefits of going to jdk8, future enhancements relevant in 2.x timeframe ( scala, dependencies) which requires it, and simplicity wrt code, test & support it looks like a good checkpoint to drop jdk7 support. As already mentioned in the thread, existing yarn clusters are unaffected if they want to continue running jdk7 and yet use spark2 (install jdk8 on all nodes and use it via JAVA_HOME, or worst case distribute jdk8 as archive - suboptimal).I am unsure about mesos (standalone might be easier upgrade I guess ?). Proposal is for 1.6x line to continue to be supported with critical fixes; newer features will require 2.x and so jdk8 Regards Mridul On Thursday, March 24, 2016, Marcelo Vanzin <van...@cloudera.com> wrote: On Thu, Mar 24, 2016 at 4:50 PM, Reynold Xin <r...@databricks.com> wrote: > If you want to go down that route, you should also ask somebody who has had > experience managing a large organization's applications and try to update > Scala version. I understand both sides. But if you look at what I've been asking since the beginning, it's all about the cost and benefits of dropping support for java 1.7. The biggest argument in your original e-mail is about testing. And the testing cost is much bigger for supporting scala 2.10 than it is for supporting java 1.7. If you read one of my earlier replies, it should be even possible to just do everything in a single job - compile for java 7 and still be able to test things in 1.8, including lambdas, which seems to be the main thing you were worried about. > On Thu, Mar 24, 2016 at 4:48 PM, Marcelo Vanzin <van...@cloudera.com> wrote: >> >> On Thu, Mar 24, 2016 at 4:46 PM, Reynold Xin <r...@databricks.com> wrote: >> > Actually it's *way* harder to upgrade Scala from 2.10 to 2.11, than >> > upgrading the JVM runtime from 7 to 8, because Scala 2.10 and 2.11 are >> > not >> > binary compatible, whereas JVM 7 and 8 are binary compatible except >> > certain >> > esoteric cases. >> >> True, but ask anyone who manages a large cluster how long it would >> take them to upgrade the jdk across their cluster and validate all >> their applications and everything... binary compatibility is a tiny >> drop in that bucket. >> >> -- >> Marcelo > > -- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
