Hey Wei-Chiu, Thanks for sharing your plans on Hadoop 3. Good to know.
1. So I understand people don't care about Hadoop 3.0 any more. I see we don't even list this line on the Releases page: https://hadoop.apache.org/releases.html I think it is fair to EOL 3.0 and stop making changes to it. 2. Sounds like there are still plans for Hadoop 3.1. So let's try to be more consistent with backporting fixes into branch-3.1 along with branch-3.2. Do we have a Wiki or a page reflecting active and EOL releases? Thanks, --Konstantin On Fri, Aug 9, 2019 at 4:06 PM Wei-Chiu Chuang <weic...@apache.org> wrote: > Erik, > I've just got this message now. Something funky is going on with my > workplace mailbox. > > With my Apache's hat on, I find it too costly to maintain a 3.0.x release > line: my personal time to try to backport commits, resolving conflicts. > Additionally, we would be forced to make additional 3.0.x releases if a new > CVE is discovered. > > With my Cloudera's hat on, the last downtream release, CDH6.3.0 is the last > release on Apache Hadoop 3.0.x. The next release, CDP 1.0 is on 3.1.x, and > at some point, may rebase onto 3.2.x. The timeline to rebase onto 3.2 is > unclear (at least 6 months out), and so I am still actively maintaining > this branch as much as possible. > > That is to say, assuming Cloudera's the only corporate sponsor for > branch-3.0, this branch is all but dead. > > On another note, I recently learned that Apache Spark 3 will only support > Hadoop 3.2, skipping Hadoop 3.0/3.1. I don't know how you guys think, but I > feel like that'd expedite the death of branch-3.1, which is why we are > thinking about a 3.2 rebase in CDP. > > On Fri, Aug 9, 2019 at 3:31 PM Erik Krogen <xkro...@apache.org> wrote: > > > I'd like to revive this discussion. Today I see committers variously > > skipping backports to branch-3.0 and/or branch-3.1 (when backporting to > > branch-2, for example) and I'm concerned that we will have divergence > > between what commits are present in which branches. > > > > Wei-Chiu, is Cloudera still rebasing on top of branch-3.1? If so then it > > seems we should be continuing to actively maintain it and not skipping > any > > backports here. > > > > > If HDFS-13596 is fixed in branch-3.1/3.2, will it be possible to > > > rolling upgrade from Hadoop 2 to Hadoop 3.1/3.2? If the answer is yes, > > > I'm thinking we can drop 3.0 and consider 3.1+ only. > > > > > > I agree on this front. I don't see any reason that we can't encourage > > upgrades from 2.x to 3.1+. > > > > Should we initiate a vote to officially EOL the 3.0 line? > > > > Thanks > > Erik > > > > P.S.: Sorry if you received this email twice, I was informed by members > of > > this team that my original email was not received by most folks. > > > > On Sun, May 26, 2019 at 11:56 PM Akira Ajisaka <aajis...@apache.org> > > wrote: > > > > > Thank you for your feedback, > > > > > > > IMHO, I'd love to see HDFS-13596 < > > > https://issues.apache.org/jira/browse/HDFS-13596> fixed to make it > > > possible to rolling upgrade from Hadoop 2 to Hadoop 3.0 > > > > > > If HDFS-13596 is fixed in branch-3.1/3.2, will it be possible to > > > rolling upgrade from Hadoop 2 to Hadoop 3.1/3.2? If the answer is yes, > > > I'm thinking we can drop 3.0 and consider 3.1+ only. > > > > > > Thanks, > > > Akira > > > > > > On Wed, May 22, 2019 at 4:39 AM Ajay Kumar <ajay.ku...@cloudera.com> > > > wrote: > > > > > > > > +1 for keeping no of active branches low, specially when it is not > used > > > actively. > > > > > > > > On Mon, May 20, 2019 at 3:33 AM Steve Loughran > > > <ste...@cloudera.com.invalid> wrote: > > > >> > > > >> I've been intermittently backporting things to it, but like you say: > > not > > > >> getting much active use. > > > >> > > > >> > > > >> 1. I'd like to view the 3.1 branch as the main "ready to play > with" > > > >> Hadoop 3.x release, with 3.2 some new features. > > > >> 2. I'm planning on backporting some of the hadoop 3.x ABFS and > S3A > > > work > > > >> to that 3.x release, and for S3A, some to 3.1.x (and indeed, > maybe > > we > > > >> should do the abfs connector, After all: it's not going to cause > > any > > > >> regressions). > > > >> 3. The hadoop-aws changes will include HADOOP-16117, AWS SDK > > update - > > > >> Sean Mackrory has been advocating "keep all active branches > current > > > with > > > >> the AWS SDKs", and I've come to agree. The testing there didn't > > find > > > any > > > >> regressions, which was a pleasant surprise. > > > >> 4. For S3A, a big patch has just gone in, HADOOP-16085, which > adds > > > etag > > > >> and version columns to the S3Guard DDB tables, and uses these at > > > load time. > > > >> Even if we don't backport that patch to the 3.1 line, it makes > > sense > > > for > > > >> all 3.1.x clients to be updating the DB with the relevant columns > > as > > > they > > > >> write it, so that on a mixed-client deployment, everyone keeps > the > > > table up > > > >> to date > > > >> > > > >> > > > >> As usual, help welcome, especially with testing > > > >> > > > >> -steve > > > >> > > > >> On Fri, May 17, 2019 at 12:15 PM Wei-Chiu Chuang < > weic...@apache.org> > > > wrote: > > > >> > > > >> > Thanks for initiating the discussion, Akira. > > > >> > > > > >> > I've given similar thoughts on the possible EOL of Hadoop > branch-3.0 > > > line. > > > >> > IMHO, I'd love to see HDFS-13596 > > > >> > <https://issues.apache.org/jira/browse/HDFS-13596> fixed to make > it > > > >> > possible to rolling upgrade from Hadoop 2 to Hadoop 3.0, and then > > roll > > > >> > another maint. release before we declare EOL. > > > >> > > > > >> > But in all seriousness, with my Cloudera hat on, CDH will soon > > rebase > > > onto > > > >> > branch-3.1 so it's unlikely Cloudera can sponsor another 3.0 maint > > > release. > > > >> > With my Apache hat on, Apache is an all-volunteer organization and > > we > > > all > > > >> > act individually, but I am just being realistic that there would > not > > > be > > > >> > much motivation to roll another release in the future. > > > >> > > > > >> > Thoughts? > > > >> > > > > >> > On Fri, May 17, 2019 at 8:31 AM Akira Ajisaka < > aajis...@apache.org> > > > wrote: > > > >> > > > > >> > > Hi folks, > > > >> > > > > > >> > > In branch-3.0, it is almost a year since 3.0.3 was released. Is > > > there > > > >> > > any committer who wants to be a release manager of 3.0.4? > > > >> > > > > > >> > > If there are no users running Apache Hadoop 3.0.x in production, > > I'm > > > >> > > thinking we can stop maintaining branch-3.0. Please let me know > > > there > > > >> > > are any users running Apache Hadoop 3.0.x. > > > >> > > > > > >> > > Any thoughts? > > > >> > > > > > >> > > -Akira > > > >> > > > > > >> > > > > > --------------------------------------------------------------------- > > > >> > > To unsubscribe, e-mail: > common-dev-unsubscr...@hadoop.apache.org > > > >> > > For additional commands, e-mail: > > common-dev-h...@hadoop.apache.org > > > >> > > > > > >> > > > > > >> > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > > > > > > > >
