Re: Compatibility Yarn 2.4 with HDFS 2.0

2014-06-08 Thread Steve Loughran
the 2.0.5 release was always considered alpha -there were some explicitly added mandatory fields to HDFS, lots of java level changes too. One of the reasons for that was to have a stable API from 2.2+, which is why 2.2 clients can work with 2.3 and 2.4 clusters today -and later versions of hadoop 2

Re: Compatibility Yarn 2.4 with HDFS 2.0

2014-06-06 Thread 张鹏
Also I tested using hdfs client 2.0 access hdfs server 2.4, this also got no compatibility: "Incorrect header or version mismatch from 10.2.201.245:59310 got version 7 expected version 9" Does this mean if we update our production cluster from 2.0 to 2.4, all clients must be re-build? Any suggest

Re: Compatibility Yarn 2.4 with HDFS 2.0

2014-06-06 Thread 张鹏
You mean YARN will call some interfaces of FileSystem that not existed in 2.0? Because I think YARN and MR depend on FileSystem interface of hadoop-common. I want to hack as below: ---> HDFS 2.0 > Hadoop-common 2.0 / YARN and MR

Re: Compatibility Yarn 2.4 with HDFS 2.0

2014-06-06 Thread Yu Azuryy
Even if you done this, it's unstable, because Yarn2.4 added some IPC interfaces. then HDFS MUST has related interfaces. On Fri, Jun 6, 2014 at 4:22 PM, 张鹏 wrote: > Hi all, > > I want to upgrade to Yarn 2.4 only. But when it access Hdfs, IPC Server > version is mismached. > > HDFS (IPC v7) cant