I ran a few compatibility tests between 2.10.0 and 3.3.0 (trunk) Unfortunately, I ran into the following problem:
Running with 2.10 RM and 3.3.0 (trunk) NM fails attempts with the following error: 2019-10-26 15:44:06,885 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RPC$VersionMismatch): Protocol org.apache.hadoop.mapred.TaskUmbilicalProtocol version mismatch. (client = 19, server = 21) The AM happened to launch on the 3.3.0 node. Is this a protobuf issue? I thought we addressed that? -Eric Payne On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <jyhung2...@gmail.com> wrote: Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and 2.10.0 clients and datanodes. Everything worked as expected. Jonathan Hung On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <ebad...@verizonmedia.com> wrote: > Hi Jonathan, > > Thanks for putting this RC together. You stated that there are > improvements related to rolling upgrades from 2.x to 3.x and I know I have > seen multiple JIRAs getting committed to that effect. Could you describe > any tests that you have done to verify rolling upgrade compatibility > for 3.x servers talking to 2.x clients and vice versa? > > Thanks, > > Eric > > On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jyhung2...@gmail.com> > wrote: > >> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar >> (HDFS-14667). Since this is the first of a minor release, we would like to >> get it into 2.10.0. >> >> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1 >> shortly. >> >> Jonathan Hung >> >> >> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote: >> >> > Thanks for the effort, Jonathan! >> > >> > +1 (non-binding) on RC0. >> > - Set up a single node cluster with the binary tarball >> > - Run Spark Pi and pySpark job >> > >> > BR, >> > Zhankun >> > >> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.had...@gmail.com >> > >> > wrote: >> > >> >> +1 on RC0. >> >> - Verified signatures >> >> - Built from sources >> >> - Ran unit tests for new features >> >> - Checked artifacts on Nexus, made sure the sources are present. >> >> >> >> Thanks >> >> --Konstantin >> >> >> >> >> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jyhung2...@gmail.com> >> >> wrote: >> >> >> >> > Hi folks, >> >> > >> >> > This is the first release candidate for the first release of Apache >> >> Hadoop >> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It >> includes >> >> > features such as: >> >> > >> >> > - User-defined resource types >> >> > - Native GPU support as a schedulable resource type >> >> > - Consistent reads from standby node >> >> > - Namenode port based selective encryption >> >> > - Improvements related to rolling upgrade support from 2.x to 3.x >> >> > >> >> > The RC0 artifacts are at: >> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/ >> >> > >> >> > RC tag is release-2.10.0-RC0. >> >> > >> >> > The maven artifacts are hosted here: >> >> > >> >> >> https://repository.apache.org/content/repositories/orgapachehadoop-1241/ >> >> > >> >> > My public key is available here: >> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS >> >> > >> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at >> 6:00 pm >> >> > PDT. >> >> > >> >> > Thanks, >> >> > Jonathan Hung >> >> > >> >> > [1] >> >> > >> >> > >> >> >> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0) >> >> > >> >> >> > >> > --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
