Hello all, please remove me from the message thread. I ave stopped working on hadoop. Thank you On Mar 10, 2015 2:45 AM, "Steve Loughran" <ste...@hortonworks.com> wrote:
> > If 3.x is going to be Java 8 & not backwards compatible, I don't expect > anyone wanting to use this in production until some time deep into 2016. > > Issue: JDK 8 vs 7 > > It will require Hadoop clusters to move up to Java 8. While there's dev > pull for this, there's ops pull against this: people are still in the > moving-off Java 6 phase due to that "it's working, don't update it" > philosophy. Java 8 is compelling to us coders, but that doesn't mean ops > want it. > > You can run JDK-8 code in a YARN cluster running on Hadoop 2.7 *today*, > the main thing is setting up JAVA_HOME. That's something we could make > easier somehow (maybe some min Java version field in resource requests that > will let apps say java 8, java 9, ...). YARN could not only set up JVM > paths, it could fail-fast if a Java version wasn't available. > > What we can't do in hadoop coretoday is set javac.version=1.8 & use java > 8 code. Downstream code ca do that (Hive, etc); they just need to accept > that they don't get to play on JDK7 clusters if they embrace l-expressions. > > So...we need to stay on java 7 for some time due to ops pull; downstream > apps get to choose what they want. We can/could enhance YARN to make JVM > choice more declarative. > > Issue: Incompatible changes > > Without knowing what is proposed for "an incompatible classpath change", I > can't say whether this is something that could be made optional. If it > isn't, then it is a python-3 class option, "rewrite your code" event, which > is going to be particularly traumatic to things like Hive that already do > complex CP games. I'm currently against any mandatory change here, though > would love to see an optional one. And if optional, it ceases to become an > incompatible change... > > Issue: Getting trunk out the door > > The main diff from branch-2 and trunk is currently the bash script > changes. These don't break client apps. May or may not break bigtop & other > downstream hadoop stacks, but developers don't need to worry about this: > no recompilation necessary > > Proposed: ship trunk as a 2.x release, compatible with JDK7 & Java code. > > It seems to me that I could go > > git checkout trunk > mvn versions:set -DnewVersion=2.8.0-SNAPSHOT > > We'd then have a version of Hadoop-trunk we could ship later this year, > compatible at the JDK and API level with the existing java code & JDK7+ > clusters. > > A classpath fix that is optional/compatible can then go out on the 2.x > line, saving the 3.x tag for something that really breaks things, forces > all downstream apps to set up new hadoop profiles, have separate modules & > generally hate the hadoop dev team > > This lets us tick off the "recent trunk release" and "fixed shell scripts" > items, pushing out those benefits to people sooner rather than later, and > puts off the "Hello, we've just broken your code" event for another 12+ > months. > > Comments? > > -Steve > > > >
