One other thing I wanted to bring up regarding HDFS-8791, we haven't backported the parallel DN upgrade improvement (HDFS-8578) to branch-2.6. HDFS-8578 is a very important related fix since otherwise upgrade will be very slow.
On Thu, Mar 31, 2016 at 10:35 AM, Andrew Wang <andrew.w...@cloudera.com> wrote: > As I expressed on HDFS-8791, I do not want to include this JIRA in a > maintenance release. I've only seen it crop up on a handful of our > customer's clusters, and large users like Twitter and Yahoo that seem to be > more affected are also the most able to patch this change in themselves. > > Layout upgrades are quite disruptive, and I don't think it's worth > breaking upgrade and downgrade expectations when it doesn't affect the (in > my experience) vast majority of users. > > Vinod seemed to have a similar opinion in his comment on HDFS-8791, but > will let him elaborate. > > Best, > Andrew > > On Thu, Mar 31, 2016 at 9:11 AM, Sean Busbey <bus...@cloudera.com> wrote: > >> As of 2 days ago, there were already 135 jiras associated with 2.7.3, >> if *any* of them end up introducing a regression the inclusion of >> HDFS-8791 means that folks will have cluster downtime in order to back >> things out. If that happens to any substantial number of downstream >> folks, or any particularly vocal downstream folks, then it is very >> likely we'll lose the remaining trust of operators for rolling out >> maintenance releases. That's a pretty steep cost. >> >> Please do not include HDFS-8791 in any 2.6.z release. Folks having to >> be aware that an upgrade from e.g. 2.6.5 to 2.7.2 will fail is an >> unreasonable burden. >> >> I agree that this fix is important, I just think we should either cut >> a version of 2.8 that includes it or find a way to do it that gives an >> operational path for rolling downgrade. >> >> On Thu, Mar 31, 2016 at 10:10 AM, Junping Du <j...@hortonworks.com> wrote: >> > Thanks for bringing up this topic, Sean. >> > When I released our latest Hadoop release 2.6.4, the patch of HDFS-8791 >> haven't been committed in so that's why we didn't discuss this earlier. >> > I remember in JIRA discussion, we treated this layout change as a >> Blocker bug that fixing a significant performance regression before but not >> a normal performance improvement. And I believe HDFS community already did >> their best with careful and patient to deliver the fix and other related >> patches (like upgrade fix in HDFS-8578). Take an example of HDFS-8578, you >> can see 30+ rounds patch review back and forth by senior committers, not to >> mention the outstanding performance test data in HDFS-8791. >> > I would trust our HDFS committers' judgement to land HDFS-8791 on >> 2.7.3. However, that needs Vinod's final confirmation who serves as RM for >> branch-2.7. In addition, I didn't see any blocker issue to bring it into >> 2.6.5 now. >> > Just my 2 cents. >> > >> > Thanks, >> > >> > Junping >> > >> > ________________________________________ >> > From: Sean Busbey <bus...@cloudera.com> >> > Sent: Thursday, March 31, 2016 2:57 PM >> > To: hdfs-dev@hadoop.apache.org >> > Cc: Hadoop Common; yarn-...@hadoop.apache.org; >> mapreduce-...@hadoop.apache.org >> > Subject: Re: 2.7.3 release plan >> > >> > A layout change in a maintenance release sounds very risky. I saw some >> > discussion on the JIRA about those risks, but the consensus seemed to >> > be "we'll leave it up to the 2.6 and 2.7 release managers." I thought >> > we did RMs per release rather than per branch? No one claiming to be a >> > release manager ever spoke up AFAICT. >> > >> > Should this change be included? Should it go into a special 2.8 >> > release as mentioned in the ticket? >> > >> > On Thu, Mar 31, 2016 at 1:45 AM, Akira AJISAKA >> > <ajisa...@oss.nttdata.co.jp> wrote: >> >> Thank you Vinod! >> >> >> >> FYI: 2.7.3 will be a bit special release. >> >> >> >> HDFS-8791 bumped up the datanode layout version, >> >> so rolling downgrade from 2.7.3 to 2.7.[0-2] >> >> is impossible. We can rollback instead. >> >> >> >> https://issues.apache.org/jira/browse/HDFS-8791 >> >> >> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html >> >> >> >> Regards, >> >> Akira >> >> >> >> >> >> On 3/31/16 08:18, Vinod Kumar Vavilapalli wrote: >> >>> >> >>> Hi all, >> >>> >> >>> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out >> (which >> >>> did go out mid February). Got a little busy since. >> >>> >> >>> Following up the 2.7.2 maintenance release, we should work towards a >> >>> 2.7.3. The focus obviously is to have blocker issues [1], bug-fixes >> and *no* >> >>> features / improvements. >> >>> >> >>> I hope to cut an RC in a week - giving enough time for outstanding >> blocker >> >>> / critical issues. Will start moving out any tickets that are not >> blockers >> >>> and/or won’t fit the timeline - there are 3 blockers and 15 critical >> tickets >> >>> outstanding as of now. >> >>> >> >>> Thanks, >> >>> +Vinod >> >>> >> >>> [1] 2.7.3 release blockers: >> >>> https://issues.apache.org/jira/issues/?filter=12335343 >> >>> >> >> >> > >> > >> > >> > -- >> > busbey >> >> >> >> -- >> busbey >> > >
