Thanks Steven for your details response. Things are more clear to me now. A follow up Qs - Looks like most of the security support depends on Hadoop ? What happens if anyone wants to use Flink with Hadoop (in a cluster where Hadoop is not there) ?
Regards, Sourav On Sun, Jan 10, 2016 at 12:41 PM, Stephan Ewen <se...@apache.org> wrote: > Hi Sourav! > > There is user-authentication support in Flink via the Hadoop / Kerberos > infrastructure. If you run Flink on YARN, it should seamlessly work that > Flink acquires the Kerberos tokens of the user that submits programs, and > authenticate itself at YARN, HDFS, and HBase with that. > > If you run Flink standalone, Flink can still authenticate at HDFS/HBase > via Kerberos, with a bit of manual help by the user (running kinit on the > workers). > > With Kafka 0.9 and Flink's upcoming connector ( > https://github.com/apache/flink/pull/1489), streaming programs can > authenticate themselves as stream brokers via SSL (and read via encrypted > connections). > > > What we have on the roadmap for the coming months it the following: > - Encrypt in-flight data streams that are exchanged between worker nodes > (TaskManagers). > - Encrypt the coordination messages between client/master/workers. > Note that these refer to encryption between Flink's own components only, > which would use transient keys generated just for a specific job or session > (hence would not need any user involvement). > > > Let us know if that answers your questions, and if that meets your > requirements. > > Greetings, > Stephan > > > On Fri, Jan 8, 2016 at 3:23 PM, Sourav Mazumder < > sourav.mazumde...@gmail.com> wrote: > >> Hi, >> >> Can anyone point me to ant documentation on support for Security in Flink >> ? >> >> The type of information I'm looking for are - >> >> 1. How do I do user level authentication to ensure that a job is >> submitted/deleted/modified by the right user ? Is it possible though the >> web client ? >> 2. Authentication across multiple slave nodes (where the task managers >> are running) and driver program so that they can communicate with each other >> 3. Support for SSL/encryption for data exchanged happening across the >> slave nodes >> 4. Support for pluggable authentication with existing solution like LDAP >> >> If not there today is there a roadmap for these security features ? >> >> Regards, >> Sourav >> > >
