Hi Robert, Thanks for the detailed response. I worked on the encryption of HDFS as well as the crypto file system in HDFS, so, I am aware of how it is done in Hadoop. Let me sync up with Max to get started on it. I will also start looking into the current implementations. Niraj
From: Robert Metzger [mailto:rmetz...@apache.org] Sent: Thursday, February 26, 2015 3:11 PM To: dev@flink.apache.org Cc: Rai, Niraj Subject: Re: Contributing to Flink Hi Niraj, Welcome to the Flink community ;) I'm really excited that you want to contribute to our project, and since you've asked for something in the security area, I actually have something very concrete in mind. We recently added some support for accessing (Kerberos) secured HDFS clusters in Flink: https://issues.apache.org/jira/browse/FLINK-1504. However, the implementation is very simple because it assumes that every worker of Flink (TaskManager) is authenticated with Kerberos (kinit). Its not very practical for large setups because you have to ssh to all machines to log into Kerberos. What I would really like to have in Flink would be an way to transfer the authentication tokens form the JobManager (master) to the TaskManagers. This way, users only have to be authenticated with Kerberos at the JobManager, and Flink is taking care of the rest. As far as I understood it, Hadoop has already all the utilities in place for getting and transferring the delegation tokens. Max Michels, another committer in our project has quite a good understanding of the details there. It would be great if you (Max) could chime in if I forgot something. If you are interested in working on this, you can file a JIRA (https://issues.apache.org/jira/browse/FLINK) for tracking the progress and discussing the details. If not I'm sure we'll come up with more interesting ideas. Robert On Thu, Feb 26, 2015 at 11:07 PM, Henry Saputra <henry.sapu...@gmail.com<mailto:henry.sapu...@gmail.com>> wrote: Hi Niraj, Thanks for your interest at Apache Flink. The quickest is to just give Flink a spin and figure out how it works. This would get you start on how it works before actually doing work on Flink =) Please do visit Flink how to contribute page [1] and subscribe to dev mailing list [2] to start following up. Welcome =) [1] http://flink.apache.org/how-to-contribute.html [2] http://flink.apache.org/community.html#mailing-lists On Thu, Feb 26, 2015 at 1:45 PM, Rai, Niraj <niraj....@intel.com<mailto:niraj....@intel.com>> wrote: > Hi Flink Dev, > I am looking to contribute to Flink, especially in the area of security. In > the past, I have contributed to Pig, Hive and HDFS. I would really > appreciate, if I can get some work assigned to me. Looking forward to hear > back from the development community of Flink. > Thanks > Niraj >
