Re: Another thought on client-side support of HDFS federation

2016-05-27 Thread Chris Nauroth
Hello Tianyi HE, I noticed that a similar design for a federation proxying model has just been proposed on Apache JIRA HDFS-10467. You might want to join the conversation there. https://issues.apache.org/jira/browse/HDFS-10467 --Chris Nauroth On 5/2/16, 10:32 AM, "Colin McCabe" wrote: >H

Re: Another thought on client-side support of HDFS federation

2016-05-02 Thread Colin McCabe
Hi Tianyi HE, Thanks for sharing this! This reminds me of the httpfs daemon. This daemon basically sits in front of an HDFS cluster and accepts requests, which it serves by forwarding them to the underlying HDFS instance. There is some documentation about it here: https://hadoop.apache.org/docs