Hi Stefano, Sounds great. Please go ahead! Note that Flink already provides the proposed feature for per-job Yarn clusters. However, it is a valuable addition to realize this feature for the Yarn session.
The only blocker that I can think of is probably this PR which changes a lot of the Yarn classes: https://github.com/apache/flink/pull/1741 There are also changes planned for the client side to decouple the Yarn support from the job submission process and make it easier to integrate other frameworks (like Mesos). I don't think that will block your contribution since a lot of the logic is probably going to be contained in separate classes which can be integrated even when code changes. Let's just stay in sync. If you like, you could start off by opening an issue and submitting a short design document. Cheers, Max On Wed, Mar 23, 2016 at 3:55 PM, Stefano Baghino <stefano.bagh...@radicalbit.io> wrote: > Hello everybody, > > some of us at Radicalbit spent the last few weeks experimenting to improve > the understanding of the compatibility of Flink with secure cluster > environments and with Kerberos in particular. > > We’ve found a possible area of improvement and would like to work on it as > part of our effort to contribute to Flink in the open: after a few tests > and a short exchange on the user mailing list > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kerberos-on-YARN-delegation-or-proxying-tp5315p5318.html> > we’ve come to realize that currently a long-running session on YARN acts on > behalf of the user that originally ran the session, not the one who’s > submitting the job. > > We think it would be a nice improvement to be able to have a single Flink > session that keeps running with several users submitting their jobs with > their own credentials. > > We’d like to develop, test and document this improvement. > > Do you think this is feasible? Are there any blockers we should be aware of > before undertaking this task? Would this be something of interest for the > community? Are there any other ongoing efforts that aim toward this? > > We’d love to have the feedback of the community on this, thank you in > advance to anyone who’s willing to share their insight and opinion with us. > > -- > BR, > Stefano Baghino > > Software Engineer @ Radicalbit
