Re: Proposal: YARN session per-job Kerberos authentication

2016-03-25 Thread Stefano Baghino
Hi Robert and Max, thanks for giving us some feedback to ponder on. We'll start by opening the issue and some sub-tasks to keep track of the prototyping and and design phase, along with development, testing and documentation. Robert, I thought of the encryption issue and was one of the things I h

Re: Proposal: YARN session per-job Kerberos authentication

2016-03-24 Thread Robert Metzger
Hi Stefano, I think the proposed feature is not limited to YARN sessions. With the code in place, also standalone clusters would allow us to authenticate file system access with the user who submitted the job. I would recommend you to do some prototyping and come up with a design document first.

Re: Proposal: YARN session per-job Kerberos authentication

2016-03-23 Thread Maximilian Michels
Hi Stefano, Sounds great. Please go ahead! Note that Flink already provides the proposed feature for per-job Yarn clusters. However, it is a valuable addition to realize this feature for the Yarn session. The only blocker that I can think of is probably this PR which changes a lot of the Yarn cla

Proposal: YARN session per-job Kerberos authentication

2016-03-23 Thread Stefano Baghino
Hello everybody, some of us at Radicalbit spent the last few weeks experimenting to improve the understanding of the compatibility of Flink with secure cluster environments and with Kerberos in particular. We’ve found a possible area of improvement and would like to work on it as part of our effo