On Wed, Mar 4, 2015 at 10:46 AM, Stack <st...@duboce.net> wrote: > In general +1 on 3.0.0. Its time. If we start now, it might make it out by > 2016. If we start now, downstreamers can start aligning themselves to land > versions that suit at about the same time. > > While two big items have been called out as possible incompatible changes, > and there is ongoing discussion as to whether they are or not*, is there > any chance of getting a longer list of big differences between the > branches? In particular I'd be interested in improvements that are 'off' by > default that would be better defaulted 'on'. > > Thanks, > St.Ack > > * Let me note that 'compatible' around these parts is a trampled concept > seemingly open to interpretation with a definition that is other than > prevails elsewhere in software. See Allen's list above, and in our > downstream project, the recent HBASE-13149 "HBase server MR tools are > broken on Hadoop 2.5+ Yarn", among others. Let 3.x be incompatible with > 2.x if only so we can leave behind all current notions of 'compatibility' > and just start over (as per Allen). >
Unfortunately, our compatibility policies <http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html> are rather loose and allow for changes that break downstream projects. Fixing the classpath issues would let us tighten our policies and bring our "compatibility store" more inline with the general expectations. > > > On Mon, Mar 2, 2015 at 3:19 PM, Andrew Wang <andrew.w...@cloudera.com> > wrote: > > > Hi devs, > > > > It's been a year and a half since 2.x went GA, and I think we're about > due > > for a 3.x release. > > Notably, there are two incompatible changes I'd like to call out, that > will > > have a tremendous positive impact for our users. > > > > First, classpath isolation being done at HADOOP-11656, which has been a > > long-standing request from many downstreams and Hadoop users. > > > > Second, bumping the source and target JDK version to JDK8 (related to > > HADOOP-11090), which is important since JDK7 is EOL in April 2015 (two > > months from now). In the past, we've had issues with our dependencies > > discontinuing support for old JDKs, so this will future-proof us. > > > > Between the two, we'll also have quite an opportunity to clean up and > > upgrade our dependencies, another common user and developer request. > > > > I'd like to propose that we start rolling a series of monthly-ish series > of > > 3.0 alpha releases ASAP, with myself volunteering to take on the RM and > > other cat herding responsibilities. There are already quite a few changes > > slated for 3.0 besides the above (for instance the shell script rewrite) > so > > there's already value in a 3.0 alpha, and the more time we give > downstreams > > to integrate, the better. > > > > This opens up discussion about inclusion of other changes, but I'm hoping > > to freeze incompatible changes after maybe two alphas, do a beta (with no > > further incompat changes allowed), and then finally a 3.x GA. For those > > keeping track, that means a 3.x GA in about four months. > > > > I would also like to stress though that this is not intended to be a big > > bang release. For instance, it would be great if we could maintain wire > > compatibility between 2.x and 3.x, so rolling upgrades work. Keeping > > branch-2 and branch-3 similar also makes backports easier, since we're > > likely maintaining 2.x for a while yet. > > > > Please let me know any comments / concerns related to the above. If > people > > are friendly to the idea, I'd like to cut a branch-3 and start working on > > the first alpha. > > > > Best, > > Andrew > > > -- Karthik Kambatla Software Engineer, Cloudera Inc. -------------------------------------------- http://five.sentenc.es
