Hi Sourav! There is user-authentication support in Flink via the Hadoop / Kerberos infrastructure. If you run Flink on YARN, it should seamlessly work that Flink acquires the Kerberos tokens of the user that submits programs, and authenticate itself at YARN, HDFS, and HBase with that.
If you run Flink standalone, Flink can still authenticate at HDFS/HBase via Kerberos, with a bit of manual help by the user (running kinit on the workers). With Kafka 0.9 and Flink's upcoming connector ( https://github.com/apache/flink/pull/1489), streaming programs can authenticate themselves as stream brokers via SSL (and read via encrypted connections). What we have on the roadmap for the coming months it the following: - Encrypt in-flight data streams that are exchanged between worker nodes (TaskManagers). - Encrypt the coordination messages between client/master/workers. Note that these refer to encryption between Flink's own components only, which would use transient keys generated just for a specific job or session (hence would not need any user involvement). Let us know if that answers your questions, and if that meets your requirements. Greetings, Stephan On Fri, Jan 8, 2016 at 3:23 PM, Sourav Mazumder <sourav.mazumde...@gmail.com > wrote: > Hi, > > Can anyone point me to ant documentation on support for Security in Flink ? > > The type of information I'm looking for are - > > 1. How do I do user level authentication to ensure that a job is > submitted/deleted/modified by the right user ? Is it possible though the > web client ? > 2. Authentication across multiple slave nodes (where the task managers are > running) and driver program so that they can communicate with each other > 3. Support for SSL/encryption for data exchanged happening across the > slave nodes > 4. Support for pluggable authentication with existing solution like LDAP > > If not there today is there a roadmap for these security features ? > > Regards, > Sourav >
