I think that 2.1 should include a visible deprecation message about Java 7, Scala 2.10 and older Hadoop versions (plus python if there is a consensus on that), to give users / admins early warning, followed by dropping them from trunk for 2.2 once 2.1 is released. Personally, we use only Scala 2.11 on JDK8. Cody - Scala 2.12 will likely be released before Spark 2.1, maybe even later this week: http://scala-lang.org/news/2.12.0-RC2
Ofir Manor Co-Founder & CTO | Equalum Mobile: +972-54-7801286 | Email: ofir.ma...@equalum.io On Tue, Oct 25, 2016 at 7:28 PM, Cody Koeninger <c...@koeninger.org> wrote: > I think only supporting 1 version of scala at any given time is not > sufficient, 2 probably is ok. > > I.e. don't drop 2.10 before 2.12 is out + supported > > On Tue, Oct 25, 2016 at 10:56 AM, Sean Owen <so...@cloudera.com> wrote: > > The general forces are that new versions of things to support emerge, and > > are valuable to support, but have some cost to support in addition to old > > versions. And the old versions become less used and therefore less > valuable > > to support, and at some point it tips to being more cost than value. It's > > hard to judge these costs and benefits. > > > > Scala is perhaps the trickiest one because of the general mutual > > incompatibilities across minor versions. The cost of supporting multiple > > versions is high, and a third version is about to arrive. That's probably > > the most pressing question. It's actually biting with some regularity > now, > > with compile errors on 2.10. > > > > (Python I confess I don't have an informed opinion about.) > > > > Java, Hadoop are not as urgent because they're more backwards-compatible. > > Anecdotally, I'd be surprised if anyone today would "upgrade" to Java 7 > or > > an old Hadoop version. And I think that's really the question. Even if > one > > decided to drop support for all this in 2.1.0, it would not mean people > > can't use Spark with these things. It merely means they can't necessarily > > use Spark 2.1.x. This is why we have maintenance branches for 1.6.x, > 2.0.x. > > > > Tying Scala 2.11/12 support to Java 8 might make sense. > > > > In fact, I think that's part of the reason that an update in master, > perhaps > > 2.1.x, could be overdue, because it actually is just the beginning of the > > end of the support burden. If you want to stop dealing with these in ~6 > > months they need to stop being supported in minor branches by right about > > now. > > > > > > > > > > On Tue, Oct 25, 2016 at 4:47 PM Mark Hamstra <m...@clearstorydata.com> > > wrote: > >> > >> What's changed since the last time we discussed these issues, about 7 > >> months ago? Or, another way to formulate the question: What are the > >> threshold criteria that we should use to decide when to end Scala 2.10 > >> and/or Java 7 support? > >> > >> On Tue, Oct 25, 2016 at 8:36 AM, Sean Owen <so...@cloudera.com> wrote: > >>> > >>> I'd like to gauge where people stand on the issue of dropping support > for > >>> a few things that were considered for 2.0. > >>> > >>> First: Scala 2.10. We've seen a number of build breakages this week > >>> because the PR builder only tests 2.11. No big deal at this stage, > but, it > >>> did cause me to wonder whether it's time to plan to drop 2.10 support, > >>> especially with 2.12 coming soon. > >>> > >>> Next, Java 7. It's reasonably old and out of public updates at this > >>> stage. It's not that painful to keep supporting, to be honest. It would > >>> simplify some bits of code, some scripts, some testing. > >>> > >>> Hadoop versions: I think the the general argument is that most anyone > >>> would be using, at the least, 2.6, and it would simplify some code > that has > >>> to reflect to use not-even-that-new APIs. It would remove some moderate > >>> complexity in the build. > >>> > >>> > >>> "When" is a tricky question. Although it's a little aggressive for > minor > >>> releases, I think these will all happen before 3.x regardless. 2.1.0 > is not > >>> out of the question, though coming soon. What about ... 2.2.0? > >>> > >>> > >>> Although I tend to favor dropping support, I'm mostly asking for > current > >>> opinions. > >> > >> > > > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
