Thanks Andrew for the huge effort. +1 (non-binding) - Downloaded binary tarball and verified md5 - Ran RM HA and verified manual failover - Verified add/remove/update scheduler configuration API (CLI/REST) works for leveldb/zookeeper backend - Verified scheduler configuration changes persisted on restart/failover - Verified "yarn rmadmin -refreshQueues" works when scheduler configuration API disabled, and does not work when scheduler configuration API enabled
Jonathan Hung On Tue, Dec 12, 2017 at 5:44 PM, Junping Du <j...@hortonworks.com> wrote: > Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get > chance to validate new RC now. > > Basically, I found two critical issues with the same rolling upgrade > scenario as where HADOOP-15059 get found previously: > HDFS-12920, we changed value format for some hdfs configurations that old > version MR client doesn't understand when fetching these configurations. > Some quick workarounds are to add old value (without time unit) in > hdfs-site.xml to override new default values but will generate many > annoying warnings. I provided my fix suggestions on the JIRA already for > more discussion. > The other one is YARN-7646. After we workaround HDFS-12920, will hit the > issue that old version MR AppMaster cannot communicate with new version of > YARN RM - could be related to resource profile changes from YARN side but > root cause are still in investigation. > > The first issue may not belong to a blocker given we can workaround this > without code change. I am not sure if we can workaround 2nd issue so far. > If not, we may have to fix this or compromise with withdrawing support of > rolling upgrade or calling it a stable release. > > > Thanks, > > Junping > > ________________________________________ > From: Robert Kanter <rkan...@cloudera.com> > Sent: Tuesday, December 12, 2017 3:10 PM > To: Arun Suresh > Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron T. > Myers; common-...@hadoop.apache.org; hdfs-dev@hadoop.apache.org; > yarn-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org > Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1 > > +1 (binding) > > + Downloaded the binary release > + Deployed on a 3 node cluster on CentOS 7.3 > + Ran some MR jobs, clicked around the UI, etc > + Ran some CLI commands (yarn logs, etc) > > Good job everyone on Hadoop 3! > > > - Robert > > On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <asur...@apache.org> wrote: > > > +1 (binding) > > > > - Verified signatures of the source tarball. > > - built from source - using the docker build environment. > > - set up a pseudo-distributed test cluster. > > - ran basic HDFS commands > > - ran some basic MR jobs > > > > Cheers > > -Arun > > > > On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <andrew.w...@cloudera.com> > > wrote: > > > > > Hi everyone, > > > > > > As a reminder, this vote closes tomorrow at 12:31pm, so please give it > a > > > whack if you have time. There are already enough binding +1s to pass > this > > > vote, but it'd be great to get additional validation. > > > > > > Thanks to everyone who's voted thus far! > > > > > > Best, > > > Andrew > > > > > > > > > > > > On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <l...@cloudera.com> wrote: > > > > > > > +1 (binding) > > > > > > > > * Verified src tarball and bin tarball, verified md5 of each. > > > > * Build source with -Pdist,native > > > > * Started a pseudo cluster > > > > * Run ec -listPolicies / -getPolicy / -setPolicy on / , and run hdfs > > > > dfs put/get/cat on "/" with XOR-2-1 policy. > > > > > > > > Thanks Andrew for this great effort! > > > > > > > > Best, > > > > > > > > > > > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang < > andrew.w...@cloudera.com > > > > > > > wrote: > > > > > Hi Wei-Chiu, > > > > > > > > > > The patchprocess directory is left over from the create-release > > > process, > > > > > and it looks empty to me. We should still file a create-release > JIRA > > to > > > > fix > > > > > this, but I think this is not a blocker. Would you agree? > > > > > > > > > > Best, > > > > > Andrew > > > > > > > > > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang < > > weic...@cloudera.com > > > > > > > > > wrote: > > > > > > > > > >> Hi Andrew, thanks the tremendous effort. > > > > >> I found an empty "patchprocess" directory in the source tarball, > > that > > > is > > > > >> not there if you clone from github. Any chance you might have some > > > > leftover > > > > >> trash when you made the tarball? > > > > >> Not wanting to nitpicking, but you might want to double check so > we > > > > don't > > > > >> ship anything private to you in public :) > > > > >> > > > > >> > > > > >> > > > > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar < > > > ajay.ku...@hortonworks.com > > > > > > > > > >> wrote: > > > > >> > > > > >>> +1 (non-binding) > > > > >>> Thanks for driving this, Andrew Wang!! > > > > >>> > > > > >>> - downloaded the src tarball and verified md5 checksum > > > > >>> - built from source with jdk 1.8.0_111-b14 > > > > >>> - brought up a pseudo distributed cluster > > > > >>> - did basic file system operations (mkdir, list, put, cat) and > > > > >>> confirmed that everything was working > > > > >>> - Run word count, pi and DFSIOTest > > > > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked > > > > >>> > > > > >>> Cheers, > > > > >>> Ajay > > > > >>> > > > > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <x...@cloudera.com> wrote: > > > > >>> > > > > >>> +1 (binding) > > > > >>> > > > > >>> - downloaded src tarball, verified md5 > > > > >>> - built from source with jdk1.8.0_112 > > > > >>> - started a pseudo cluster with hdfs and kms > > > > >>> - sanity checked encryption related operations working > > > > >>> - sanity checked webui and logs. > > > > >>> > > > > >>> -Xiao > > > > >>> > > > > >>> On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers < > > a...@apache.org> > > > > >>> wrote: > > > > >>> > > > > >>> > +1 (binding) > > > > >>> > > > > > >>> > - downloaded the src tarball and built the source (-Pdist > > > > -Pnative) > > > > >>> > - verified the checksum > > > > >>> > - brought up a secure pseudo distributed cluster > > > > >>> > - did some basic file system operations (mkdir, list, put, > > cat) > > > > and > > > > >>> > confirmed that everything was working > > > > >>> > - confirmed that the web UI worked > > > > >>> > > > > > >>> > Best, > > > > >>> > Aaron > > > > >>> > > > > > >>> > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang < > > > > >>> andrew.w...@cloudera.com> > > > > >>> > wrote: > > > > >>> > > > > > >>> > > Hi all, > > > > >>> > > > > > > >>> > > Let me start, as always, by thanking the efforts of all > the > > > > >>> contributors > > > > >>> > > who contributed to this release, especially those who > > jumped > > > on > > > > >>> the > > > > >>> > issues > > > > >>> > > found in RC0. > > > > >>> > > > > > > >>> > > I've prepared RC1 for Apache Hadoop 3.0.0. This release > > > > >>> incorporates 302 > > > > >>> > > fixed JIRAs since the previous 3.0.0-beta1 release. > > > > >>> > > > > > > >>> > > You can find the artifacts here: > > > > >>> > > > > > > >>> > > http://home.apache.org/~wang/3.0.0-RC1/ > > > > >>> > > > > > > >>> > > I've done the traditional testing of building from the > > source > > > > >>> tarball and > > > > >>> > > running a Pi job on a single node cluster. I also > verified > > > that > > > > >>> the > > > > >>> > shaded > > > > >>> > > jars are not empty. > > > > >>> > > > > > > >>> > > Found one issue that create-release (probably due to the > > mvn > > > > >>> deploy > > > > >>> > change) > > > > >>> > > didn't sign the artifacts, but I fixed that by calling > mvn > > > one > > > > >>> more time. > > > > >>> > > Available here: > > > > >>> > > > > > > >>> > > https://repository.apache.org/ > > content/repositories/orgapache > > > > >>> hadoop-1075/ > > > > >>> > > > > > > >>> > > This release will run the standard 5 days, closing on Dec > > > 13th > > > > at > > > > >>> 12:31pm > > > > >>> > > Pacific. My +1 to start. > > > > >>> > > > > > > >>> > > Best, > > > > >>> > > Andrew > > > > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> ------------------------------------------------------------ > > > --------- > > > > >>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > > > >>> For additional commands, e-mail: common-dev-help@hadoop.apache. > org > > > > >>> > > > > >> > > > > >> > > > > >> > > > > >> > > > > > > > > > > > > > > > > -- > > > > Lei (Eddy) Xu > > > > Software Engineer, Cloudera > > > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > >
