Re: [DISCUSS] Merge HDFS-10467 to (Router-based federation) trunk

2017-08-31 Thread Iñigo Goiri
Manoj, thanks for the comments. I added the consolidated branch patch to HDFS-10467 (CC Brahma). Regarding the comparison with existing approaches, I'd say that the real comparison is ViewFs (already in the docs). This is complementary to the current HDFS federation; you have multiple namespaces

Re: [DISCUSS] Merge HDFS-10467 to (Router-based federation) trunk

2017-08-31 Thread Iñigo Goiri
Agreed on this not being the cleanest.. Just filed it this morning: HDFS-12384. On Thu, Aug 31, 2017 at 4:36 PM, Andrew Wang wrote: > v) mvn install (and package) is failing with following error >> >> [INFO] Adding ignore: * >> [WARNING] Rule 1: org.apache.maven.plugins.enforcer.BanDuplicateC

Re: [DISCUSS] Merge HDFS-10467 to (Router-based federation) trunk

2017-08-31 Thread Andrew Wang
> > v) mvn install (and package) is failing with following error > > [INFO] Adding ignore: * > [WARNING] Rule 1: org.apache.maven.plugins.enforcer.BanDuplicateClasses > failed with message: > Duplicate classes found: > > Found in: > org.apache.hadoop:hadoop-client-minicluster:jar:3.0.0- > b

Re: [DISCUSS] Merge HDFS-10467 to (Router-based federation) trunk

2017-08-29 Thread Manoj Govindassamy
>>* >> >> >> >> org.apache.hadoop >> hadoop-client-minicluster >> >>* >>

Re: [DISCUSS] Merge HDFS-10467 to (Router-based federation) trunk

2017-08-28 Thread Iñigo Goiri
* > > > > > Please correct me If I am wrong. > > > --Brahma Reddy Battula > > -Original Message- > From: Chris Douglas [mailto:cdoug...@apache.org] > Sent: 25 August 2017 06:37 > To: Andrew Wang > Cc: Iñigo Go

RE: [DISCUSS] Merge HDFS-10467 to (Router-based federation) trunk

2017-08-28 Thread Brahma Reddy Battula
oug...@apache.org] Sent: 25 August 2017 06:37 To: Andrew Wang Cc: Iñigo Goiri; hdfs-dev@hadoop.apache.org; su...@apache.org Subject: Re: [DISCUSS] Merge HDFS-10467 to (Router-based federation) trunk On Thu, Aug 24, 2017 at 2:25 PM, Andrew Wang wrote: > Do you mind holding this until 3.1? Same re

Re: [DISCUSS] Merge HDFS-10467 to (Router-based federation) trunk

2017-08-24 Thread Chris Douglas
On Thu, Aug 24, 2017 at 2:25 PM, Andrew Wang wrote: > Do you mind holding this until 3.1? Same reasoning as for the other branch > merge proposals, we're simply too late in the 3.0.0 release cycle. That wouldn't be too dire. That said, this has the same design and impact as YARN federation. Spec

Re: [DISCUSS] Merge HDFS-10467 to (Router-based federation) trunk

2017-08-24 Thread Andrew Wang
Do you mind holding this until 3.1? Same reasoning as for the other branch merge proposals, we're simply too late in the 3.0.0 release cycle. On Thu, Aug 24, 2017 at 1:39 PM, Chris Douglas wrote: > I'd definitely support merging this to trunk. The implementation is > almost entirely outside of H

Re: [DISCUSS] Merge HDFS-10467 to (Router-based federation) trunk

2017-08-24 Thread Chris Douglas
I'd definitely support merging this to trunk. The implementation is almost entirely outside of HDFS and, as Inigo detailed, has been tested at scale. The branch is in a functional state with documentation and tests. -C On Mon, Aug 21, 2017 at 6:11 PM, Iñigo Goiri wrote: > Hi all, > > > > We would

[DISCUSS] Merge HDFS-10467 to (Router-based federation) trunk

2017-08-21 Thread Iñigo Goiri
Hi all, We would like to open a discussion on merging the Router-based Federation feature to trunk. Last week, there was a thread about which branches would go into 3.0 and given that YARN federation is going, this might be a good time for this to be merged too. We have been running "Router-b