Hi Gabor, I haven't found time to look into the updated FLIP yet. I'll try to do it asap.
Cheers, Till On Wed, Jun 16, 2021 at 9:35 PM Konstantin Knauf <kna...@apache.org> wrote: > Hi Gabor, > > > However representing Kerberos as completely new feature is not true > because > it's already in since Flink makes authentication at least with HDFS and > Hbase through Kerberos. > > True, that is one way to look at it, but there are differences, too: > Control Plane vs Data Plane, Core vs Connectors. > > > Adding OIDC or OAuth2 has the exact same concerns what you've guys just > raised. Why exactly these? If you think this would be beneficial we can > discuss it in detail > > That's exactly my point. Once we start adding authx support, we will > sooner or later discuss other options besides Kerberos, too. A user who > would like to use OAuth can not easily use Kerberos, right? > That is one of the reasons I am skeptical about adding initial authx > support. > > > Related authorization you've mentioned it can be complicated over time. > Can > you show us an example? We've knowledge with couple of open source > components > but authorization was never a horror complex story. I personally have the > most experience with Spark which I think is quite simple and stable. Users > can be viewers/admins > and jobs started by others can't be modified. If you can share an example > over-complication we can discuss on facts. > > Authorization is a new aspect that needs to be considered for every > addition to the REST API. In the future users might ask for additional > roles (e.g. an editor), user-defined roles and you've already mentioned > job-level permissions yourself. And keep in mind that there might also be > larger additions in the future like the flink-sql-gateway. Contributions > like this become more expensive the more aspects we need to consider. > > In general, I believe, it is important that the community focuses its > efforts where we can generate the most value to the user and - personally - > I don't think there is much to gain by extending Flink's scope in that > direction. Of course, this is not black and white and there are other valid > opinions. > > Thanks, > > Konstantin > > On Wed, Jun 16, 2021 at 7:38 PM Gabor Somogyi <gabor.g.somo...@gmail.com> > wrote: > >> Hi Konstantin, >> >> Thanks for the response. Related new feature introduction in case of Basic >> auth I tend to agree, anything else can be chosen. >> >> However representing Kerberos as completely new feature is not true >> because >> it's already in since Flink makes authentication at least with HDFS and >> Hbase through Kerberos. >> The main problem with the actual Kerberos implementation is that it >> contains several bugs and only partially implemented. Following your >> suggestion can we agree that we >> skip the Basic auth implementation and finish an already started Kerberos >> story by adding History Server and Job Dashboard authentication? >> >> Adding OIDC or OAuth2 has the exact same concerns what you've guys just >> raised. Why exactly these? If you think this would be beneficial we can >> discuss it in detail >> but as a side story it would be good to finish a halfway done Kerberos >> story. >> >> Related authorization you've mentioned it can be complicated over time. >> Can >> you show us an example? We've knowledge with couple of open source >> components >> but authorization was never a horror complex story. I personally have the >> most experience with Spark which I think is quite simple and stable. Users >> can be viewers/admins >> and jobs started by others can't be modified. If you can share an example >> over-complication we can discuss on facts. >> >> Thank you in advance! >> >> BR, >> G >> >> >> On Wed, Jun 16, 2021 at 5:42 PM Konstantin Knauf <kna...@apache.org> >> wrote: >> >> > Hi everyone, >> > >> > sorry for joining late and thanks for the insightful discussion. >> > >> > In general, I'd personally prefer not to increase the surface area of >> > Apache Flink unless there is a good reason. It seems we all agree that >> > authx is not part of the core value proposition of Apache Flink, so if >> we >> > can delegate this problem to a more specialized tool, I am in favor of >> > that. Apache Flink is already huge and a lot of work goes into >> maintenance, >> > so I personally have become more sensitive to this aspect over time. >> > >> > If we add support for Basic Auth and Kerberos now, users will sooner or >> > later ask for OIDC, LDAP, SAML,... I acknowledge that Kerberos is widely >> > used in the corporate, on-premises context, but isn't the focus moving >> more >> > towards more web-friendly standards like OIDC/OAuth 2.0? If we only >> want to >> > support a single protocol, there is an argument to be made that it >> should >> > be OIDC and Dex [1,2] as a bridge to everything else. Have OIDC or >> OAuth2 >> > been considered instead of Kerberos? How do you see the market moving? >> But >> > as I said before, in my opinion we can generate more value by investing >> > into other areas of Apache Flink. >> > >> > Authorization also has the potential to become more fine-grained and >> > complex over time: you already mentioned restricting the actions that a >> > specific user can do in a cluster. >> > >> > Cheers, >> > >> > Konstantin >> > >> > [1] https://github.com/dexidp/dex >> > [2] https://github.com/dexidp/dex/issues/1903 >> > >> > >> > On Wed, Jun 16, 2021 at 11:44 AM Gabor Somogyi < >> gabor.g.somo...@gmail.com> >> > wrote: >> > >> >> Hi Till, >> >> >> >> Did you have the chance to take a look at the doc? Not yet seen any >> >> update. >> >> >> >> BR, >> >> G >> >> >> >> >> >> On Wed, Jun 9, 2021 at 1:43 PM Till Rohrmann <trohrm...@apache.org> >> >> wrote: >> >> >> >> > Thanks for the update Gabor. I'll take a look and respond in the >> >> document. >> >> > >> >> > Cheers, >> >> > Till >> >> > >> >> > On Wed, Jun 9, 2021 at 12:59 PM Gabor Somogyi < >> >> gabor.g.somo...@gmail.com> >> >> > wrote: >> >> > >> >> >> Hi Till, >> >> >> >> >> >> Your proxy suggestion has been considered in-depth and updated the >> FLIP >> >> >> accordingly. >> >> >> We've considered 2 proxy implementation (Nginx and Squid) but >> according >> >> >> to our analysis and testing it's not suitable for the mentioned >> >> use-cases. >> >> >> Please take a look at the rejected alternatives for detailed >> >> explanation. >> >> >> >> >> >> Thanks for your time in advance! >> >> >> >> >> >> BR, >> >> >> G >> >> >> >> >> >> >> >> >> On Fri, Jun 4, 2021 at 3:31 PM Till Rohrmann <trohrm...@apache.org> >> >> >> wrote: >> >> >> >> >> >>> As I've said I am not a security expert and that's why I have to >> ask >> >> for >> >> >>> clarification, Gabor. You are saying that if we configure a >> >> truststore for >> >> >>> the REST endpoint with a single trusted certificate which has been >> >> >>> generated by the operator of the Flink cluster, then the attacker >> can >> >> >>> generate a new certificate, sign it and then talk to the Flink >> >> cluster if >> >> >>> he has access to the node on which the REST endpoint runs? My >> >> understanding >> >> >>> was that you need the corresponding private key which in my >> proposed >> >> setup >> >> >>> would be under the control of the operator as well (e.g. stored in >> a >> >> >>> keystore on the same machine but guarded by some secret). That way >> >> (if I am >> >> >>> not mistaken), only the entity which has access to the keystore is >> >> able to >> >> >>> talk to the Flink cluster. >> >> >>> >> >> >>> Maybe we are also getting our wires crossed here and are talking >> about >> >> >>> different things. >> >> >>> >> >> >>> Thanks for listing the pros and cons of Kerberos. Concerning what >> >> other >> >> >>> authentication mechanisms are used in the industry, I am not 100% >> >> sure. >> >> >>> >> >> >>> Cheers, >> >> >>> Till >> >> >>> >> >> >>> On Fri, Jun 4, 2021 at 11:09 AM Gabor Somogyi < >> >> gabor.g.somo...@gmail.com> >> >> >>> wrote: >> >> >>> >> >> >>>> > I did not mean for the user to sign its own certificates but for >> >> the >> >> >>>> operator of the cluster. Once the user request hits the proxy, it >> >> should no >> >> >>>> longer be under his control. I think I do not fully understand yet >> >> why this >> >> >>>> would not work. >> >> >>>> I said it's not solving the authentication problem over any proxy. >> >> Even >> >> >>>> if the operator is signing the certificate one can have access to >> an >> >> >>>> internal node. >> >> >>>> Such case anybody can craft certificates which is accepted by the >> >> >>>> server. When it's accepted a bad guy can cancel jobs causing huge >> >> impacts. >> >> >>>> >> >> >>>> > Also, I am missing a bit the comparison of Kerberos to other >> >> >>>> authentication mechanisms and why they were rejected in favour of >> >> Kerberos. >> >> >>>> PROS: >> >> >>>> * Since it's not depending on cloud provider and/or k8s or >> bare-metal >> >> >>>> etc. deployment it's the biggest plus >> >> >>>> * Centralized with tools and no need to write tons of tools around >> >> >>>> * There are clients/tools on almost all OS-es and several >> languages >> >> >>>> * Super huge users are using it for years in production w/o huge >> >> issues >> >> >>>> * Provides cross-realm trust possibility amongst other features >> >> >>>> * Several open source components using it which could increase >> >> >>>> compatibility >> >> >>>> >> >> >>>> CONS: >> >> >>>> * Not everybody using kerberos >> >> >>>> * It would increase the code footprint but this is true for many >> >> >>>> features (as a side note I'm here to maintain it) >> >> >>>> >> >> >>>> Feel free to add your points because it only represents a single >> >> >>>> viewpoint. >> >> >>>> Also if you have any better option for strong authentication >> please >> >> >>>> share it and we can consider the pros/cons here. >> >> >>>> >> >> >>>> BR, >> >> >>>> G >> >> >>>> >> >> >>>> >> >> >>>> On Fri, Jun 4, 2021 at 10:32 AM Till Rohrmann < >> trohrm...@apache.org> >> >> >>>> wrote: >> >> >>>> >> >> >>>>> I did not mean for the user to sign its own certificates but for >> the >> >> >>>>> operator of the cluster. Once the user request hits the proxy, it >> >> should no >> >> >>>>> longer be under his control. I think I do not fully understand >> yet >> >> why this >> >> >>>>> would not work. >> >> >>>>> >> >> >>>>> What I would like to avoid is to add more complexity into Flink >> if >> >> >>>>> there is an easy solution which fulfills the requirements. That's >> >> why I >> >> >>>>> would like to exercise thoroughly through the different >> >> alternatives. Also, >> >> >>>>> I am missing a bit the comparison of Kerberos to other >> >> authentication >> >> >>>>> mechanisms and why they were rejected in favour of Kerberos. >> >> >>>>> >> >> >>>>> Cheers, >> >> >>>>> Till >> >> >>>>> >> >> >>>>> On Fri, Jun 4, 2021 at 10:26 AM Gyula Fóra <gyf...@apache.org> >> >> wrote: >> >> >>>>> >> >> >>>>>> Hi! >> >> >>>>>> >> >> >>>>>> I think there might be possible alternatives but it seems >> Kerberos >> >> on >> >> >>>>>> the rest endpoint ticks all the right boxes and provides a super >> >> clean and >> >> >>>>>> simple solution for strong authentication. >> >> >>>>>> >> >> >>>>>> I wouldn’t even consider sidecar proxies etc if we can solve it >> in >> >> >>>>>> such a simple way as proposed by G. >> >> >>>>>> >> >> >>>>>> Cheers >> >> >>>>>> Gyula >> >> >>>>>> >> >> >>>>>> On Fri, 4 Jun 2021 at 10:03, Till Rohrmann < >> trohrm...@apache.org> >> >> >>>>>> wrote: >> >> >>>>>> >> >> >>>>>>> I am not saying that we shouldn't add a strong authentication >> >> >>>>>>> mechanism if there are good reasons for it. I primarily would >> >> like to >> >> >>>>>>> understand the context a bit better in order to give qualified >> >> feedback and >> >> >>>>>>> come to a good decision. In order to do this, I have the >> feeling >> >> that we >> >> >>>>>>> haven't fully considered all available options which are on the >> >> table, tbh. >> >> >>>>>>> >> >> >>>>>>> Does the problem of certificate expiry also apply for >> self-signed >> >> >>>>>>> certificates? If yes, then this should then also be a problem >> for >> >> the >> >> >>>>>>> internal encryption of Flink's communication. If not, then one >> >> could use >> >> >>>>>>> self-signed certificates with a longer validity to solve the >> >> mentioned >> >> >>>>>>> issue. >> >> >>>>>>> >> >> >>>>>>> I think you can set up Flink in such a way that you don't have >> to >> >> >>>>>>> handle all the different certificates. For example, you could >> >> deploy Flink >> >> >>>>>>> with a "sidecar proxy" which is responsible for the >> >> authentication using an >> >> >>>>>>> arbitrary method (e.g. Kerberos) and then bind the REST >> endpoint >> >> to a local >> >> >>>>>>> network interface. That way, the REST endpoint would only be >> >> available >> >> >>>>>>> through the sidecar proxy. Additionally, one could enable SSL >> for >> >> this >> >> >>>>>>> communication. Would this be a solution for the problem? >> >> >>>>>>> >> >> >>>>>>> Cheers, >> >> >>>>>>> Till >> >> >>>>>>> >> >> >>>>>>> On Thu, Jun 3, 2021 at 10:46 PM Márton Balassi < >> >> >>>>>>> balassi.mar...@gmail.com> wrote: >> >> >>>>>>> >> >> >>>>>>>> That is an interesting idea, Till. >> >> >>>>>>>> >> >> >>>>>>>> The main issue with it is that TLS certificates have an >> >> expiration >> >> >>>>>>>> time, usually they get approved for a couple years. Forcing >> our >> >> users to >> >> >>>>>>>> restart jobs to reprovision TLS certificates would be weird >> when >> >> we could >> >> >>>>>>>> just implement a single proper strong authentication mechanism >> >> instead in a >> >> >>>>>>>> couple hundred lines of code. :-) >> >> >>>>>>>> >> >> >>>>>>>> In many cases it is also impractical to go the TLS mutual >> route, >> >> >>>>>>>> because the Flink Dashboard can end up on any node in the >> >> k8s/Yarn cluster >> >> >>>>>>>> which means that we need a certificate per node (due to the >> >> mutual auth), >> >> >>>>>>>> but if we also want to protect the private key of these from >> >> users >> >> >>>>>>>> accidentally or intentionally leaking them then we need this >> per >> >> user. As >> >> >>>>>>>> in we end up managing user*machine number certificates and >> >> having to renew >> >> >>>>>>>> them periodically, which albeit automatable is unfortunately >> not >> >> yet >> >> >>>>>>>> automated in all large organizations. >> >> >>>>>>>> >> >> >>>>>>>> I fully agree that TLS certificate mutual authentication has >> its >> >> >>>>>>>> nice properties, especially at very large (multiple thousand >> >> node) clusters >> >> >>>>>>>> - but it has its own challenges too. Thanks for bringing it >> up. >> >> >>>>>>>> >> >> >>>>>>>> Happy to have this added to the rejected alternative list so >> that >> >> >>>>>>>> we have the full picture documented. >> >> >>>>>>>> >> >> >>>>>>>> On Thu, Jun 3, 2021 at 5:52 PM Till Rohrmann < >> >> trohrm...@apache.org> >> >> >>>>>>>> wrote: >> >> >>>>>>>> >> >> >>>>>>>>> I guess the idea would then be to let the proxy do the >> >> >>>>>>>>> authentication job and only forward the request via an SSL >> >> mutually >> >> >>>>>>>>> encrypted connection to the Flink cluster. Would this be >> >> possible? The >> >> >>>>>>>>> beauty of this setup is in my opinion that this setup should >> >> work with all >> >> >>>>>>>>> kinds of authentication mechanisms. >> >> >>>>>>>>> >> >> >>>>>>>>> Cheers, >> >> >>>>>>>>> Till >> >> >>>>>>>>> >> >> >>>>>>>>> On Thu, Jun 3, 2021 at 3:12 PM Gabor Somogyi < >> >> >>>>>>>>> gabor.g.somo...@gmail.com> wrote: >> >> >>>>>>>>> >> >> >>>>>>>>>> Thanks for giving options to fulfil the need. >> >> >>>>>>>>>> >> >> >>>>>>>>>> Users are looking for a solution where users can be >> identified >> >> on >> >> >>>>>>>>>> the whole cluster and restrict access to resources/actions. >> >> >>>>>>>>>> A good example for such an action is cancelling other users >> >> >>>>>>>>>> running jobs. >> >> >>>>>>>>>> >> >> >>>>>>>>>> * SSL does provide mutual authentication but when >> >> authentication >> >> >>>>>>>>>> passed there is no user based on restrictions can be made. >> >> >>>>>>>>>> * The less problematic part is that generating/maintaining >> >> short >> >> >>>>>>>>>> time valid certificates would be a hard (that's the reason >> KDC >> >> like servers >> >> >>>>>>>>>> exist). >> >> >>>>>>>>>> Having long time valid certificates would widen the attack >> >> >>>>>>>>>> surface but since the first concern is there this is just a >> >> cosmetic issue. >> >> >>>>>>>>>> >> >> >>>>>>>>>> All in all using TLS certificates is not sufficient in these >> >> >>>>>>>>>> environments unfortunately. >> >> >>>>>>>>>> >> >> >>>>>>>>>> BR, >> >> >>>>>>>>>> G >> >> >>>>>>>>>> >> >> >>>>>>>>>> >> >> >>>>>>>>>> On Thu, Jun 3, 2021 at 12:49 PM Till Rohrmann < >> >> >>>>>>>>>> trohrm...@apache.org> wrote: >> >> >>>>>>>>>> >> >> >>>>>>>>>>> Thanks for the information Gabor. If it is about securing >> the >> >> >>>>>>>>>>> communication between the REST client and the REST server, >> >> then Flink >> >> >>>>>>>>>>> already supports enabling mutual SSL authentication [1]. >> >> Would this be >> >> >>>>>>>>>>> enough to secure the communication and to pass an audit? >> >> >>>>>>>>>>> >> >> >>>>>>>>>>> [1] >> >> >>>>>>>>>>> >> >> >> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/security/security-ssl/#external--rest-connectivity >> >> >>>>>>>>>>> >> >> >>>>>>>>>>> Cheers, >> >> >>>>>>>>>>> Till >> >> >>>>>>>>>>> >> >> >>>>>>>>>>> On Thu, Jun 3, 2021 at 10:33 AM Gabor Somogyi < >> >> >>>>>>>>>>> gabor.g.somo...@gmail.com> wrote: >> >> >>>>>>>>>>> >> >> >>>>>>>>>>>> Hi Till, >> >> >>>>>>>>>>>> >> >> >>>>>>>>>>>> Since I'm working in security area 10+ years let me share >> my >> >> >>>>>>>>>>>> thought. >> >> >>>>>>>>>>>> I would like to emphasise there are experts better than me >> >> but >> >> >>>>>>>>>>>> I have some >> >> >>>>>>>>>>>> basics. >> >> >>>>>>>>>>>> The discussion is open and not trying to tell alone >> things... >> >> >>>>>>>>>>>> >> >> >>>>>>>>>>>> > I mean if an attacker can get access to one of the >> >> machines, >> >> >>>>>>>>>>>> then it >> >> >>>>>>>>>>>> should also be possible to obtain the right Kerberos >> token. >> >> >>>>>>>>>>>> Not necessarily. For example if one gets access to a >> specific >> >> >>>>>>>>>>>> user's >> >> >>>>>>>>>>>> credentials then it's not possible to compromise other >> user's >> >> >>>>>>>>>>>> jobs, data, >> >> >>>>>>>>>>>> etc... >> >> >>>>>>>>>>>> Security is like an onion, the more layers has been added >> the >> >> >>>>>>>>>>>> more time an >> >> >>>>>>>>>>>> attacker needs to proceed. >> >> >>>>>>>>>>>> At the end of the day if one is in, then most probably can >> >> find >> >> >>>>>>>>>>>> the way but >> >> >>>>>>>>>>>> this time is normally enough to sysadmins or security >> >> experts to >> >> >>>>>>>>>>>> close down the system and minimize the damage. >> >> >>>>>>>>>>>> >> >> >>>>>>>>>>>> The other thing is that all tokens has a timeout and if >> the >> >> >>>>>>>>>>>> token is >> >> >>>>>>>>>>>> invalid then the attacker can't proceed further. >> >> >>>>>>>>>>>> >> >> >>>>>>>>>>>> > Is Kerberos also the standard authentication protocol >> for >> >> >>>>>>>>>>>> Kubernetes >> >> >>>>>>>>>>>> deployments? >> >> >>>>>>>>>>>> Kerberos is an industry standard which is cloud/deployment >> >> >>>>>>>>>>>> agnostic and it >> >> >>>>>>>>>>>> can be used in any deployments including k8s. >> >> >>>>>>>>>>>> The main intention is to use kerberos in k8s deployments >> too >> >> >>>>>>>>>>>> since we're >> >> >>>>>>>>>>>> going this direction as well. >> >> >>>>>>>>>>>> Please see how Spark does this: >> >> >>>>>>>>>>>> >> >> >>>>>>>>>>>> >> >> >> https://spark.apache.org/docs/latest/security.html#secure-interaction-with-kubernetes >> >> >>>>>>>>>>>> >> >> >>>>>>>>>>>> Last but not least the most important reason to add at >> least >> >> >>>>>>>>>>>> one strong >> >> >>>>>>>>>>>> authentication is that we have users who has >> >> >>>>>>>>>>>> hard requirements on this. They're doing security audits >> and >> >> if >> >> >>>>>>>>>>>> they fail >> >> >>>>>>>>>>>> then it's deal breaking. >> >> >>>>>>>>>>>> That is why we have added kerberos at the first place. >> >> >>>>>>>>>>>> Unfortunately we >> >> >>>>>>>>>>>> can't name them in this public list, however >> >> >>>>>>>>>>>> the customers who specifically asked for this were mainly >> in >> >> >>>>>>>>>>>> the banking >> >> >>>>>>>>>>>> and telco sector. >> >> >>>>>>>>>>>> >> >> >>>>>>>>>>>> BR, >> >> >>>>>>>>>>>> G >> >> >>>>>>>>>>>> >> >> >>>>>>>>>>>> >> >> >>>>>>>>>>>> On Thu, Jun 3, 2021 at 9:20 AM Till Rohrmann < >> >> >>>>>>>>>>>> trohrm...@apache.org> wrote: >> >> >>>>>>>>>>>> >> >> >>>>>>>>>>>> > Thanks for updating the document Márton. Why is it that >> >> banks >> >> >>>>>>>>>>>> will >> >> >>>>>>>>>>>> > consider it more secure if Flink comes with Kerberos >> >> >>>>>>>>>>>> authentication >> >> >>>>>>>>>>>> > (assuming a properly secured setup)? I mean if an >> attacker >> >> >>>>>>>>>>>> can get access >> >> >>>>>>>>>>>> > to one of the machines, then it should also be possible >> to >> >> >>>>>>>>>>>> obtain the right >> >> >>>>>>>>>>>> > Kerberos token. >> >> >>>>>>>>>>>> > >> >> >>>>>>>>>>>> > I am not an authentication expert and that's why I >> wanted >> >> to >> >> >>>>>>>>>>>> ask what are >> >> >>>>>>>>>>>> > other authentication protocols other than Kerberos? Why >> did >> >> >>>>>>>>>>>> we select >> >> >>>>>>>>>>>> > Kerberos and not any other authentication protocol? >> Maybe >> >> you >> >> >>>>>>>>>>>> can list the >> >> >>>>>>>>>>>> > pros and cons for the different protocols. Is Kerberos >> also >> >> >>>>>>>>>>>> the standard >> >> >>>>>>>>>>>> > authentication protocol for Kubernetes deployments? If >> not, >> >> >>>>>>>>>>>> what would be >> >> >>>>>>>>>>>> > the answer when deploying on K8s? >> >> >>>>>>>>>>>> > >> >> >>>>>>>>>>>> > Cheers, >> >> >>>>>>>>>>>> > Till >> >> >>>>>>>>>>>> > >> >> >>>>>>>>>>>> > On Wed, Jun 2, 2021 at 12:07 PM Gabor Somogyi < >> >> >>>>>>>>>>>> gabor.g.somo...@gmail.com> >> >> >>>>>>>>>>>> > wrote: >> >> >>>>>>>>>>>> > >> >> >>>>>>>>>>>> >> Hi team, >> >> >>>>>>>>>>>> >> >> >> >>>>>>>>>>>> >> Happy to be here and hope I can provide quality >> additions >> >> in >> >> >>>>>>>>>>>> the future. >> >> >>>>>>>>>>>> >> >> >> >>>>>>>>>>>> >> Thank you all for helpful the suggestions! >> >> >>>>>>>>>>>> >> Considering them the FLIP has been modified and the >> work >> >> >>>>>>>>>>>> continues on the >> >> >>>>>>>>>>>> >> already existing Jira. >> >> >>>>>>>>>>>> >> >> >> >>>>>>>>>>>> >> BR, >> >> >>>>>>>>>>>> >> G >> >> >>>>>>>>>>>> >> >> >> >>>>>>>>>>>> >> >> >> >>>>>>>>>>>> >> On Wed, Jun 2, 2021 at 11:23 AM Márton Balassi < >> >> >>>>>>>>>>>> balassi.mar...@gmail.com> >> >> >>>>>>>>>>>> >> wrote: >> >> >>>>>>>>>>>> >> >> >> >>>>>>>>>>>> >>> Thanks, Chesney - I totally missed that. Answered on >> the >> >> >>>>>>>>>>>> ticket too, let >> >> >>>>>>>>>>>> >>> us continue there then. >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> >>> Till, I agree that we should keep this codepath as >> slim >> >> as >> >> >>>>>>>>>>>> possible. It >> >> >>>>>>>>>>>> >>> is an important design decision that we aim to keep >> the >> >> >>>>>>>>>>>> list of >> >> >>>>>>>>>>>> >>> authentication protocols to a minimum. We believe that >> >> this >> >> >>>>>>>>>>>> should not be a >> >> >>>>>>>>>>>> >>> primary concern of Flink and a trusted proxy service >> (for >> >> >>>>>>>>>>>> example Apache >> >> >>>>>>>>>>>> >>> Knox) should be used to enable a multitude of enduser >> >> >>>>>>>>>>>> authentication >> >> >>>>>>>>>>>> >>> mechanisms. The bare minimum of authentication >> mechanisms >> >> >>>>>>>>>>>> to support >> >> >>>>>>>>>>>> >>> consequently consist of a single strong authentication >> >> >>>>>>>>>>>> protocol for which >> >> >>>>>>>>>>>> >>> Kerberos is the enterprise solution and HTTP Basic >> >> primary >> >> >>>>>>>>>>>> for development >> >> >>>>>>>>>>>> >>> and light-weight scenarios. >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> >>> Added the above wording to G's doc. >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> >> >> >> https://docs.google.com/document/d/1NMPeJ9H0G49TGy3AzTVVJVKmYC0okwOtqLTSPnGqzHw/edit >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> >>> On Tue, Jun 1, 2021 at 11:47 AM Chesnay Schepler < >> >> >>>>>>>>>>>> ches...@apache.org> >> >> >>>>>>>>>>>> >>> wrote: >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> >>>> There's a related effort: >> >> >>>>>>>>>>>> >>>> https://issues.apache.org/jira/browse/FLINK-21108 >> >> >>>>>>>>>>>> >>>> >> >> >>>>>>>>>>>> >>>> On 6/1/2021 10:14 AM, Till Rohrmann wrote: >> >> >>>>>>>>>>>> >>>> > Hi Gabor, welcome to the Flink community! >> >> >>>>>>>>>>>> >>>> > >> >> >>>>>>>>>>>> >>>> > Thanks for sharing this proposal with the community >> >> >>>>>>>>>>>> Márton. In >> >> >>>>>>>>>>>> >>>> general, I >> >> >>>>>>>>>>>> >>>> > agree that authentication is missing and that this >> is >> >> >>>>>>>>>>>> required for >> >> >>>>>>>>>>>> >>>> using >> >> >>>>>>>>>>>> >>>> > Flink within an enterprise. The thing I am >> wondering >> >> is >> >> >>>>>>>>>>>> whether this >> >> >>>>>>>>>>>> >>>> > feature strictly needs to be implemented inside of >> >> Flink >> >> >>>>>>>>>>>> or whether a >> >> >>>>>>>>>>>> >>>> proxy >> >> >>>>>>>>>>>> >>>> > setup could do the job? Have you considered this >> >> option? >> >> >>>>>>>>>>>> If yes, then >> >> >>>>>>>>>>>> >>>> it >> >> >>>>>>>>>>>> >>>> > would be good to list it under the point of >> rejected >> >> >>>>>>>>>>>> alternatives. >> >> >>>>>>>>>>>> >>>> > >> >> >>>>>>>>>>>> >>>> > I do see the benefit of implementing this feature >> >> inside >> >> >>>>>>>>>>>> of Flink if >> >> >>>>>>>>>>>> >>>> many >> >> >>>>>>>>>>>> >>>> > users need it. If not, then it might be easier for >> the >> >> >>>>>>>>>>>> project to not >> >> >>>>>>>>>>>> >>>> > increase the surface area since it makes the >> overall >> >> >>>>>>>>>>>> maintenance >> >> >>>>>>>>>>>> >>>> harder. >> >> >>>>>>>>>>>> >>>> > >> >> >>>>>>>>>>>> >>>> > Cheers, >> >> >>>>>>>>>>>> >>>> > Till >> >> >>>>>>>>>>>> >>>> > >> >> >>>>>>>>>>>> >>>> > On Mon, May 31, 2021 at 4:57 PM Márton Balassi < >> >> >>>>>>>>>>>> mbala...@apache.org> >> >> >>>>>>>>>>>> >>>> wrote: >> >> >>>>>>>>>>>> >>>> > >> >> >>>>>>>>>>>> >>>> >> Hi team, >> >> >>>>>>>>>>>> >>>> >> >> >> >>>>>>>>>>>> >>>> >> Firstly I would like to introduce Gabor or G [1] >> for >> >> >>>>>>>>>>>> short to the >> >> >>>>>>>>>>>> >>>> >> community, he is a Spark committer who has >> recently >> >> >>>>>>>>>>>> transitioned to >> >> >>>>>>>>>>>> >>>> the >> >> >>>>>>>>>>>> >>>> >> Flink Engineering team at Cloudera and is looking >> >> >>>>>>>>>>>> forward to >> >> >>>>>>>>>>>> >>>> contributing >> >> >>>>>>>>>>>> >>>> >> to Apache Flink. Previously G primarily focused on >> >> >>>>>>>>>>>> Spark Streaming >> >> >>>>>>>>>>>> >>>> and >> >> >>>>>>>>>>>> >>>> >> security. >> >> >>>>>>>>>>>> >>>> >> >> >> >>>>>>>>>>>> >>>> >> Based on requests from our customers G has >> >> implemented >> >> >>>>>>>>>>>> Kerberos and >> >> >>>>>>>>>>>> >>>> HTTP >> >> >>>>>>>>>>>> >>>> >> Basic Authentication for the Flink Dashboard and >> >> >>>>>>>>>>>> HistoryServer. >> >> >>>>>>>>>>>> >>>> Previously >> >> >>>>>>>>>>>> >>>> >> lacked an authentication story. >> >> >>>>>>>>>>>> >>>> >> >> >> >>>>>>>>>>>> >>>> >> We are looking to contribute this functionality >> back >> >> to >> >> >>>>>>>>>>>> the >> >> >>>>>>>>>>>> >>>> community, we >> >> >>>>>>>>>>>> >>>> >> believe that given Flink's maturity there should >> be a >> >> >>>>>>>>>>>> common code >> >> >>>>>>>>>>>> >>>> solution >> >> >>>>>>>>>>>> >>>> >> for this general pattern. >> >> >>>>>>>>>>>> >>>> >> >> >> >>>>>>>>>>>> >>>> >> We are looking forward to your feedback on G's >> >> design. >> >> >>>>>>>>>>>> [2] >> >> >>>>>>>>>>>> >>>> >> >> >> >>>>>>>>>>>> >>>> >> [1] http://gaborsomogyi.com/ >> >> >>>>>>>>>>>> >>>> >> [2] >> >> >>>>>>>>>>>> >>>> >> >> >> >>>>>>>>>>>> >>>> >> >> >> >>>>>>>>>>>> >>>> >> >> >>>>>>>>>>>> >> >> >> https://docs.google.com/document/d/1NMPeJ9H0G49TGy3AzTVVJVKmYC0okwOtqLTSPnGqzHw/edit >> >> >>>>>>>>>>>> >>>> >> >> >> >>>>>>>>>>>> >>>> >> >> >>>>>>>>>>>> >>>> >> >> >>>>>>>>>>>> >> >> >>>>>>>>>>> >> >> >> > >> > >> > -- >> > >> > Konstantin Knauf >> > >> > https://twitter.com/snntrable >> > >> > https://github.com/knaufk >> > >> > > > -- > > Konstantin Knauf > > https://twitter.com/snntrable > > https://github.com/knaufk >
