Thanks @Eric Payne <erichadoo...@yahoo.com> Due to another issue, we have to spin an RC1. In this case, could we revert the patch which caused this problem.
Cancelling this RC0. Thanks to everyone who voted. I will spin RC1 as soon as possible. - Sunil On Fri, Nov 30, 2018 at 4:14 AM Eric Payne <erichadoo...@yahoo.com.invalid> wrote: > The problem is not with preemption. The yarn-site.xml that I use for my > pseudo-cluster includes a second xml: > xi:include href=".../yarn-scheduler.xml" > > The property for yarn.resourcemanager.scheduler.monitor.enable = true is > in this yarn-scheduler.xml. > > This value IS READ when then RM starts. > > However, when the refreshQueues command is run, this value IS NOT READ. > > So, it looks like xml include files are not read on refresh. This will > affect any property. I just happened to notice it on the preemption > properties. > > I would like input from all of you to determine if this is a blocker for > release. I'm on the fence. > > Thanks, > -Eric > > > > > > > On Wednesday, November 28, 2018, 4:58:50 PM CST, Eric Payne < > erichadoo...@yahoo.com.INVALID> wrote: > > > > > > Sunil, > > So, the basic symptoms are that if preemption is enabled on any queue, the > preemption is disabled after a 'yarn rm -refreshQueues'. In addition, all > of the preemption-specific properties are set back to the default values. > > This was introduced in branch-3.1, so it is NOT new behavior for release > 3.2.0. I am still tracking down the cause. I will open a JIRA once I do > further investigation if there is not one already. > > This will be a problem for installations which use preemption and which > use the refreshQueues feature. > > Thanks, > -Eric > > > On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne < > eric.payne1...@yahoo.com> wrote: > > > > > > Sunil, thanks for all of the hard work on this release. > > I have discovered that queue refresh doesn't work in some cases. For > example, when I change > yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take > effect unless I restart the RM. > > I am still investigating, but I thought I should bring this up asap. > > Thanks, > -Eric > > > > > On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <sun...@apache.org> > wrote: > > > > > > Hi folks, > > > > Thanks to all contributors who helped in this release [1]. I have created > > first release candidate (RC0) for Apache Hadoop 3.2.0. > > > Artifacts for this RC are available here: > > http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/ > > > > RC tag in git is release-3.2.0-RC0. > > > > The maven artifacts are available via repository.apache.org at > > https://repository.apache.org/content/repositories/orgapachehadoop-1174/ > > > This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST. > > > > 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature > additions > > are the highlights of this release. > > 1. Node Attributes Support in YARN > > 2. Hadoop Submarine project for running Deep Learning workloads on YARN > > 3. Support service upgrade via YARN Service API and CLI > > 4. HDFS Storage Policy Satisfier > > 5. Support Windows Azure Storage - Blob file system in Hadoop > > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a > > 7. Improvements in Router-based HDFS federation > > > > Thanks to Wangda, Vinod, Marton for helping me in preparing the release. > > I have done few testing with my pseudo cluster. My +1 to start. > > > > Regards, > > Sunil > > > > [1] > > > https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E > > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0) > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved > ORDER BY fixVersion ASC > > --------------------------------------------------------------------- > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > >
