Support for Erasure Coding for Hadoop 2.7

2017-06-27 Thread Rahul Shrivastava
Hi All, I am trying to find out if erasure coding is possible for Hadoop 2.7 ( even with a backport). I have gone through some of the jira and content on google but found that current branch is with Hadoop 3.0. I have a specific questions 1. Is it possible to backport the EC to 2.7 ? thanks Rahu

How to Tag a request using optional field

2015-06-04 Thread Rahul Shrivastava
Hi, Suppose I write a Java client to create a directory on HDFS. Is there a way to tag this request and get the tagged information on NameNode via DFSInotifyEventInputStream or otherwise ? In short, is there a way to give optional information attached to every request ? I do not want to save th

How to convert an modify FSDataInputStream

2015-05-14 Thread Rahul Shrivastava
Hi All, I have a requirement whereby I have to extend the functionality of HDFS to filter out sensitive information ( SSN, Bank Account ) during data read. The solution has to be done at the API definition layer ( API of FSDataInputStream) such that it works with all our existing ETL programs. I l

Code example of Simple Authenication using Hadoop

2015-02-23 Thread Rahul Shrivastava
I am new to Hadoop Authentication ( Kerberos, Simple Authentication). I have been doing reading for security aspects of Hadoop at http://blog.godatadriven.com/kerberos_kdc_install.html and http://blog.cloudera.com/blog/2012/03/authorization-and-authentication-in-hadoop/. Can someone provide me a Ja

Re: Proxy for HDFS

2015-02-10 Thread Rahul Shrivastava
ahoo-inc.com.INVALID'> > wrote: > read the documentation. > > ________ > > From: Rahul Shrivastava > To: Rajiv Chittajallu > Cc: "hdfs-dev@hadoop.apache.org" > Sent: Monday, February 9, 2015 9:53 PM > Subject: Re: Proxy for

Re: Proxy for HDFS

2015-02-09 Thread Rahul Shrivastava
roxy. > > > [1] > http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#ACLs_Access_Control_Lists > ________ > From: Rahul Shrivastava > To: hdfs-dev@hadoop.apache.org; Rajiv Chittajallu > Sent: Monday, February 9, 201

Re: Proxy for HDFS

2015-02-09 Thread Rahul Shrivastava
should work. > > If you can share a use case, it would probably help. > > -rajive > > > - Original Message - > From: Rahul Shrivastava > To: hdfs-dev@hadoop.apache.org; Rajiv Chittajallu > Cc: > Sent: Monday, February 9, 2015 2:58 PM > Subject: Re:

Re: Proxy for HDFS

2015-02-09 Thread Rahul Shrivastava
, 2015 at 2:53 PM, Rajiv Chittajallu < raj...@yahoo-inc.com.invalid> wrote: > > https://github.com/apache/hadoop/tree/branch-2.6/hadoop-hdfs-project/hadoop-hdfs-httpfs > > > > > - Original Message - > From: Rahul Shrivastava > To: hdfs-dev@hadoop.apache.org

Proxy for HDFS

2015-02-09 Thread Rahul Shrivastava
Hi, I have looking in HDFS code base ( 2.6.0) for the last couple of days for any possible proxy that we could utilize to create a proxy for HDFS. Is there any hook within HDFS to build a proxy around that. thanks Rahul