I left some comments in the Google document. It would be great if someone from the community with security experience could also take a look at it. Maybe Eron you have an opinion on the topic.
Cheers, Till On Thu, Jun 17, 2021 at 6:57 PM Till Rohrmann <trohrm...@apache.org> wrote: > Hi Gabor, > > I haven't found time to look into the updated FLIP yet. I'll try to do it > asap. > > Cheers, > Till > > On Wed, Jun 16, 2021 at 9:35 PM Konstantin Knauf <kna...@apache.org> > wrote: > >> Hi Gabor, >> >> > However representing Kerberos as completely new feature is not true >> because >> it's already in since Flink makes authentication at least with HDFS and >> Hbase through Kerberos. >> >> True, that is one way to look at it, but there are differences, too: >> Control Plane vs Data Plane, Core vs Connectors. >> >> > Adding OIDC or OAuth2 has the exact same concerns what you've guys just >> raised. Why exactly these? If you think this would be beneficial we can >> discuss it in detail >> >> That's exactly my point. Once we start adding authx support, we will >> sooner or later discuss other options besides Kerberos, too. A user who >> would like to use OAuth can not easily use Kerberos, right? >> That is one of the reasons I am skeptical about adding initial authx >> support. >> >> > Related authorization you've mentioned it can be complicated over time. >> Can >> you show us an example? We've knowledge with couple of open source >> components >> but authorization was never a horror complex story. I personally have the >> most experience with Spark which I think is quite simple and stable. Users >> can be viewers/admins >> and jobs started by others can't be modified. If you can share an example >> over-complication we can discuss on facts. >> >> Authorization is a new aspect that needs to be considered for every >> addition to the REST API. In the future users might ask for additional >> roles (e.g. an editor), user-defined roles and you've already mentioned >> job-level permissions yourself. And keep in mind that there might also be >> larger additions in the future like the flink-sql-gateway. Contributions >> like this become more expensive the more aspects we need to consider. >> >> In general, I believe, it is important that the community focuses its >> efforts where we can generate the most value to the user and - personally - >> I don't think there is much to gain by extending Flink's scope in that >> direction. Of course, this is not black and white and there are other valid >> opinions. >> >> Thanks, >> >> Konstantin >> >> On Wed, Jun 16, 2021 at 7:38 PM Gabor Somogyi <gabor.g.somo...@gmail.com> >> wrote: >> >>> Hi Konstantin, >>> >>> Thanks for the response. Related new feature introduction in case of >>> Basic >>> auth I tend to agree, anything else can be chosen. >>> >>> However representing Kerberos as completely new feature is not true >>> because >>> it's already in since Flink makes authentication at least with HDFS and >>> Hbase through Kerberos. >>> The main problem with the actual Kerberos implementation is that it >>> contains several bugs and only partially implemented. Following your >>> suggestion can we agree that we >>> skip the Basic auth implementation and finish an already started Kerberos >>> story by adding History Server and Job Dashboard authentication? >>> >>> Adding OIDC or OAuth2 has the exact same concerns what you've guys just >>> raised. Why exactly these? If you think this would be beneficial we can >>> discuss it in detail >>> but as a side story it would be good to finish a halfway done Kerberos >>> story. >>> >>> Related authorization you've mentioned it can be complicated over time. >>> Can >>> you show us an example? We've knowledge with couple of open source >>> components >>> but authorization was never a horror complex story. I personally have the >>> most experience with Spark which I think is quite simple and stable. >>> Users >>> can be viewers/admins >>> and jobs started by others can't be modified. If you can share an example >>> over-complication we can discuss on facts. >>> >>> Thank you in advance! >>> >>> BR, >>> G >>> >>> >>> On Wed, Jun 16, 2021 at 5:42 PM Konstantin Knauf <kna...@apache.org> >>> wrote: >>> >>> > Hi everyone, >>> > >>> > sorry for joining late and thanks for the insightful discussion. >>> > >>> > In general, I'd personally prefer not to increase the surface area of >>> > Apache Flink unless there is a good reason. It seems we all agree that >>> > authx is not part of the core value proposition of Apache Flink, so if >>> we >>> > can delegate this problem to a more specialized tool, I am in favor of >>> > that. Apache Flink is already huge and a lot of work goes into >>> maintenance, >>> > so I personally have become more sensitive to this aspect over time. >>> > >>> > If we add support for Basic Auth and Kerberos now, users will sooner or >>> > later ask for OIDC, LDAP, SAML,... I acknowledge that Kerberos is >>> widely >>> > used in the corporate, on-premises context, but isn't the focus moving >>> more >>> > towards more web-friendly standards like OIDC/OAuth 2.0? If we only >>> want to >>> > support a single protocol, there is an argument to be made that it >>> should >>> > be OIDC and Dex [1,2] as a bridge to everything else. Have OIDC or >>> OAuth2 >>> > been considered instead of Kerberos? How do you see the market moving? >>> But >>> > as I said before, in my opinion we can generate more value by investing >>> > into other areas of Apache Flink. >>> > >>> > Authorization also has the potential to become more fine-grained and >>> > complex over time: you already mentioned restricting the actions that a >>> > specific user can do in a cluster. >>> > >>> > Cheers, >>> > >>> > Konstantin >>> > >>> > [1] https://github.com/dexidp/dex >>> > [2] https://github.com/dexidp/dex/issues/1903 >>> > >>> > >>> > On Wed, Jun 16, 2021 at 11:44 AM Gabor Somogyi < >>> gabor.g.somo...@gmail.com> >>> > wrote: >>> > >>> >> Hi Till, >>> >> >>> >> Did you have the chance to take a look at the doc? Not yet seen any >>> >> update. >>> >> >>> >> BR, >>> >> G >>> >> >>> >> >>> >> On Wed, Jun 9, 2021 at 1:43 PM Till Rohrmann <trohrm...@apache.org> >>> >> wrote: >>> >> >>> >> > Thanks for the update Gabor. I'll take a look and respond in the >>> >> document. >>> >> > >>> >> > Cheers, >>> >> > Till >>> >> > >>> >> > On Wed, Jun 9, 2021 at 12:59 PM Gabor Somogyi < >>> >> gabor.g.somo...@gmail.com> >>> >> > wrote: >>> >> > >>> >> >> Hi Till, >>> >> >> >>> >> >> Your proxy suggestion has been considered in-depth and updated the >>> FLIP >>> >> >> accordingly. >>> >> >> We've considered 2 proxy implementation (Nginx and Squid) but >>> according >>> >> >> to our analysis and testing it's not suitable for the mentioned >>> >> use-cases. >>> >> >> Please take a look at the rejected alternatives for detailed >>> >> explanation. >>> >> >> >>> >> >> Thanks for your time in advance! >>> >> >> >>> >> >> BR, >>> >> >> G >>> >> >> >>> >> >> >>> >> >> On Fri, Jun 4, 2021 at 3:31 PM Till Rohrmann <trohrm...@apache.org >>> > >>> >> >> wrote: >>> >> >> >>> >> >>> As I've said I am not a security expert and that's why I have to >>> ask >>> >> for >>> >> >>> clarification, Gabor. You are saying that if we configure a >>> >> truststore for >>> >> >>> the REST endpoint with a single trusted certificate which has been >>> >> >>> generated by the operator of the Flink cluster, then the attacker >>> can >>> >> >>> generate a new certificate, sign it and then talk to the Flink >>> >> cluster if >>> >> >>> he has access to the node on which the REST endpoint runs? My >>> >> understanding >>> >> >>> was that you need the corresponding private key which in my >>> proposed >>> >> setup >>> >> >>> would be under the control of the operator as well (e.g. stored >>> in a >>> >> >>> keystore on the same machine but guarded by some secret). That way >>> >> (if I am >>> >> >>> not mistaken), only the entity which has access to the keystore is >>> >> able to >>> >> >>> talk to the Flink cluster. >>> >> >>> >>> >> >>> Maybe we are also getting our wires crossed here and are talking >>> about >>> >> >>> different things. >>> >> >>> >>> >> >>> Thanks for listing the pros and cons of Kerberos. Concerning what >>> >> other >>> >> >>> authentication mechanisms are used in the industry, I am not 100% >>> >> sure. >>> >> >>> >>> >> >>> Cheers, >>> >> >>> Till >>> >> >>> >>> >> >>> On Fri, Jun 4, 2021 at 11:09 AM Gabor Somogyi < >>> >> gabor.g.somo...@gmail.com> >>> >> >>> wrote: >>> >> >>> >>> >> >>>> > I did not mean for the user to sign its own certificates but >>> for >>> >> the >>> >> >>>> operator of the cluster. Once the user request hits the proxy, it >>> >> should no >>> >> >>>> longer be under his control. I think I do not fully understand >>> yet >>> >> why this >>> >> >>>> would not work. >>> >> >>>> I said it's not solving the authentication problem over any >>> proxy. >>> >> Even >>> >> >>>> if the operator is signing the certificate one can have access >>> to an >>> >> >>>> internal node. >>> >> >>>> Such case anybody can craft certificates which is accepted by the >>> >> >>>> server. When it's accepted a bad guy can cancel jobs causing huge >>> >> impacts. >>> >> >>>> >>> >> >>>> > Also, I am missing a bit the comparison of Kerberos to other >>> >> >>>> authentication mechanisms and why they were rejected in favour of >>> >> Kerberos. >>> >> >>>> PROS: >>> >> >>>> * Since it's not depending on cloud provider and/or k8s or >>> bare-metal >>> >> >>>> etc. deployment it's the biggest plus >>> >> >>>> * Centralized with tools and no need to write tons of tools >>> around >>> >> >>>> * There are clients/tools on almost all OS-es and several >>> languages >>> >> >>>> * Super huge users are using it for years in production w/o huge >>> >> issues >>> >> >>>> * Provides cross-realm trust possibility amongst other features >>> >> >>>> * Several open source components using it which could increase >>> >> >>>> compatibility >>> >> >>>> >>> >> >>>> CONS: >>> >> >>>> * Not everybody using kerberos >>> >> >>>> * It would increase the code footprint but this is true for many >>> >> >>>> features (as a side note I'm here to maintain it) >>> >> >>>> >>> >> >>>> Feel free to add your points because it only represents a single >>> >> >>>> viewpoint. >>> >> >>>> Also if you have any better option for strong authentication >>> please >>> >> >>>> share it and we can consider the pros/cons here. >>> >> >>>> >>> >> >>>> BR, >>> >> >>>> G >>> >> >>>> >>> >> >>>> >>> >> >>>> On Fri, Jun 4, 2021 at 10:32 AM Till Rohrmann < >>> trohrm...@apache.org> >>> >> >>>> wrote: >>> >> >>>> >>> >> >>>>> I did not mean for the user to sign its own certificates but >>> for the >>> >> >>>>> operator of the cluster. Once the user request hits the proxy, >>> it >>> >> should no >>> >> >>>>> longer be under his control. I think I do not fully understand >>> yet >>> >> why this >>> >> >>>>> would not work. >>> >> >>>>> >>> >> >>>>> What I would like to avoid is to add more complexity into Flink >>> if >>> >> >>>>> there is an easy solution which fulfills the requirements. >>> That's >>> >> why I >>> >> >>>>> would like to exercise thoroughly through the different >>> >> alternatives. Also, >>> >> >>>>> I am missing a bit the comparison of Kerberos to other >>> >> authentication >>> >> >>>>> mechanisms and why they were rejected in favour of Kerberos. >>> >> >>>>> >>> >> >>>>> Cheers, >>> >> >>>>> Till >>> >> >>>>> >>> >> >>>>> On Fri, Jun 4, 2021 at 10:26 AM Gyula Fóra <gyf...@apache.org> >>> >> wrote: >>> >> >>>>> >>> >> >>>>>> Hi! >>> >> >>>>>> >>> >> >>>>>> I think there might be possible alternatives but it seems >>> Kerberos >>> >> on >>> >> >>>>>> the rest endpoint ticks all the right boxes and provides a >>> super >>> >> clean and >>> >> >>>>>> simple solution for strong authentication. >>> >> >>>>>> >>> >> >>>>>> I wouldn’t even consider sidecar proxies etc if we can solve >>> it in >>> >> >>>>>> such a simple way as proposed by G. >>> >> >>>>>> >>> >> >>>>>> Cheers >>> >> >>>>>> Gyula >>> >> >>>>>> >>> >> >>>>>> On Fri, 4 Jun 2021 at 10:03, Till Rohrmann < >>> trohrm...@apache.org> >>> >> >>>>>> wrote: >>> >> >>>>>> >>> >> >>>>>>> I am not saying that we shouldn't add a strong authentication >>> >> >>>>>>> mechanism if there are good reasons for it. I primarily would >>> >> like to >>> >> >>>>>>> understand the context a bit better in order to give qualified >>> >> feedback and >>> >> >>>>>>> come to a good decision. In order to do this, I have the >>> feeling >>> >> that we >>> >> >>>>>>> haven't fully considered all available options which are on >>> the >>> >> table, tbh. >>> >> >>>>>>> >>> >> >>>>>>> Does the problem of certificate expiry also apply for >>> self-signed >>> >> >>>>>>> certificates? If yes, then this should then also be a problem >>> for >>> >> the >>> >> >>>>>>> internal encryption of Flink's communication. If not, then one >>> >> could use >>> >> >>>>>>> self-signed certificates with a longer validity to solve the >>> >> mentioned >>> >> >>>>>>> issue. >>> >> >>>>>>> >>> >> >>>>>>> I think you can set up Flink in such a way that you don't >>> have to >>> >> >>>>>>> handle all the different certificates. For example, you could >>> >> deploy Flink >>> >> >>>>>>> with a "sidecar proxy" which is responsible for the >>> >> authentication using an >>> >> >>>>>>> arbitrary method (e.g. Kerberos) and then bind the REST >>> endpoint >>> >> to a local >>> >> >>>>>>> network interface. That way, the REST endpoint would only be >>> >> available >>> >> >>>>>>> through the sidecar proxy. Additionally, one could enable SSL >>> for >>> >> this >>> >> >>>>>>> communication. Would this be a solution for the problem? >>> >> >>>>>>> >>> >> >>>>>>> Cheers, >>> >> >>>>>>> Till >>> >> >>>>>>> >>> >> >>>>>>> On Thu, Jun 3, 2021 at 10:46 PM Márton Balassi < >>> >> >>>>>>> balassi.mar...@gmail.com> wrote: >>> >> >>>>>>> >>> >> >>>>>>>> That is an interesting idea, Till. >>> >> >>>>>>>> >>> >> >>>>>>>> The main issue with it is that TLS certificates have an >>> >> expiration >>> >> >>>>>>>> time, usually they get approved for a couple years. Forcing >>> our >>> >> users to >>> >> >>>>>>>> restart jobs to reprovision TLS certificates would be weird >>> when >>> >> we could >>> >> >>>>>>>> just implement a single proper strong authentication >>> mechanism >>> >> instead in a >>> >> >>>>>>>> couple hundred lines of code. :-) >>> >> >>>>>>>> >>> >> >>>>>>>> In many cases it is also impractical to go the TLS mutual >>> route, >>> >> >>>>>>>> because the Flink Dashboard can end up on any node in the >>> >> k8s/Yarn cluster >>> >> >>>>>>>> which means that we need a certificate per node (due to the >>> >> mutual auth), >>> >> >>>>>>>> but if we also want to protect the private key of these from >>> >> users >>> >> >>>>>>>> accidentally or intentionally leaking them then we need this >>> per >>> >> user. As >>> >> >>>>>>>> in we end up managing user*machine number certificates and >>> >> having to renew >>> >> >>>>>>>> them periodically, which albeit automatable is unfortunately >>> not >>> >> yet >>> >> >>>>>>>> automated in all large organizations. >>> >> >>>>>>>> >>> >> >>>>>>>> I fully agree that TLS certificate mutual authentication has >>> its >>> >> >>>>>>>> nice properties, especially at very large (multiple thousand >>> >> node) clusters >>> >> >>>>>>>> - but it has its own challenges too. Thanks for bringing it >>> up. >>> >> >>>>>>>> >>> >> >>>>>>>> Happy to have this added to the rejected alternative list so >>> that >>> >> >>>>>>>> we have the full picture documented. >>> >> >>>>>>>> >>> >> >>>>>>>> On Thu, Jun 3, 2021 at 5:52 PM Till Rohrmann < >>> >> trohrm...@apache.org> >>> >> >>>>>>>> wrote: >>> >> >>>>>>>> >>> >> >>>>>>>>> I guess the idea would then be to let the proxy do the >>> >> >>>>>>>>> authentication job and only forward the request via an SSL >>> >> mutually >>> >> >>>>>>>>> encrypted connection to the Flink cluster. Would this be >>> >> possible? The >>> >> >>>>>>>>> beauty of this setup is in my opinion that this setup should >>> >> work with all >>> >> >>>>>>>>> kinds of authentication mechanisms. >>> >> >>>>>>>>> >>> >> >>>>>>>>> Cheers, >>> >> >>>>>>>>> Till >>> >> >>>>>>>>> >>> >> >>>>>>>>> On Thu, Jun 3, 2021 at 3:12 PM Gabor Somogyi < >>> >> >>>>>>>>> gabor.g.somo...@gmail.com> wrote: >>> >> >>>>>>>>> >>> >> >>>>>>>>>> Thanks for giving options to fulfil the need. >>> >> >>>>>>>>>> >>> >> >>>>>>>>>> Users are looking for a solution where users can be >>> identified >>> >> on >>> >> >>>>>>>>>> the whole cluster and restrict access to resources/actions. >>> >> >>>>>>>>>> A good example for such an action is cancelling other users >>> >> >>>>>>>>>> running jobs. >>> >> >>>>>>>>>> >>> >> >>>>>>>>>> * SSL does provide mutual authentication but when >>> >> authentication >>> >> >>>>>>>>>> passed there is no user based on restrictions can be made. >>> >> >>>>>>>>>> * The less problematic part is that generating/maintaining >>> >> short >>> >> >>>>>>>>>> time valid certificates would be a hard (that's the reason >>> KDC >>> >> like servers >>> >> >>>>>>>>>> exist). >>> >> >>>>>>>>>> Having long time valid certificates would widen the attack >>> >> >>>>>>>>>> surface but since the first concern is there this is just a >>> >> cosmetic issue. >>> >> >>>>>>>>>> >>> >> >>>>>>>>>> All in all using TLS certificates is not sufficient in >>> these >>> >> >>>>>>>>>> environments unfortunately. >>> >> >>>>>>>>>> >>> >> >>>>>>>>>> BR, >>> >> >>>>>>>>>> G >>> >> >>>>>>>>>> >>> >> >>>>>>>>>> >>> >> >>>>>>>>>> On Thu, Jun 3, 2021 at 12:49 PM Till Rohrmann < >>> >> >>>>>>>>>> trohrm...@apache.org> wrote: >>> >> >>>>>>>>>> >>> >> >>>>>>>>>>> Thanks for the information Gabor. If it is about securing >>> the >>> >> >>>>>>>>>>> communication between the REST client and the REST server, >>> >> then Flink >>> >> >>>>>>>>>>> already supports enabling mutual SSL authentication [1]. >>> >> Would this be >>> >> >>>>>>>>>>> enough to secure the communication and to pass an audit? >>> >> >>>>>>>>>>> >>> >> >>>>>>>>>>> [1] >>> >> >>>>>>>>>>> >>> >> >>> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/security/security-ssl/#external--rest-connectivity >>> >> >>>>>>>>>>> >>> >> >>>>>>>>>>> Cheers, >>> >> >>>>>>>>>>> Till >>> >> >>>>>>>>>>> >>> >> >>>>>>>>>>> On Thu, Jun 3, 2021 at 10:33 AM Gabor Somogyi < >>> >> >>>>>>>>>>> gabor.g.somo...@gmail.com> wrote: >>> >> >>>>>>>>>>> >>> >> >>>>>>>>>>>> Hi Till, >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> Since I'm working in security area 10+ years let me >>> share my >>> >> >>>>>>>>>>>> thought. >>> >> >>>>>>>>>>>> I would like to emphasise there are experts better than >>> me >>> >> but >>> >> >>>>>>>>>>>> I have some >>> >> >>>>>>>>>>>> basics. >>> >> >>>>>>>>>>>> The discussion is open and not trying to tell alone >>> things... >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> > I mean if an attacker can get access to one of the >>> >> machines, >>> >> >>>>>>>>>>>> then it >>> >> >>>>>>>>>>>> should also be possible to obtain the right Kerberos >>> token. >>> >> >>>>>>>>>>>> Not necessarily. For example if one gets access to a >>> specific >>> >> >>>>>>>>>>>> user's >>> >> >>>>>>>>>>>> credentials then it's not possible to compromise other >>> user's >>> >> >>>>>>>>>>>> jobs, data, >>> >> >>>>>>>>>>>> etc... >>> >> >>>>>>>>>>>> Security is like an onion, the more layers has been >>> added the >>> >> >>>>>>>>>>>> more time an >>> >> >>>>>>>>>>>> attacker needs to proceed. >>> >> >>>>>>>>>>>> At the end of the day if one is in, then most probably >>> can >>> >> find >>> >> >>>>>>>>>>>> the way but >>> >> >>>>>>>>>>>> this time is normally enough to sysadmins or security >>> >> experts to >>> >> >>>>>>>>>>>> close down the system and minimize the damage. >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> The other thing is that all tokens has a timeout and if >>> the >>> >> >>>>>>>>>>>> token is >>> >> >>>>>>>>>>>> invalid then the attacker can't proceed further. >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> > Is Kerberos also the standard authentication protocol >>> for >>> >> >>>>>>>>>>>> Kubernetes >>> >> >>>>>>>>>>>> deployments? >>> >> >>>>>>>>>>>> Kerberos is an industry standard which is >>> cloud/deployment >>> >> >>>>>>>>>>>> agnostic and it >>> >> >>>>>>>>>>>> can be used in any deployments including k8s. >>> >> >>>>>>>>>>>> The main intention is to use kerberos in k8s deployments >>> too >>> >> >>>>>>>>>>>> since we're >>> >> >>>>>>>>>>>> going this direction as well. >>> >> >>>>>>>>>>>> Please see how Spark does this: >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> >>> >> >>> https://spark.apache.org/docs/latest/security.html#secure-interaction-with-kubernetes >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> Last but not least the most important reason to add at >>> least >>> >> >>>>>>>>>>>> one strong >>> >> >>>>>>>>>>>> authentication is that we have users who has >>> >> >>>>>>>>>>>> hard requirements on this. They're doing security audits >>> and >>> >> if >>> >> >>>>>>>>>>>> they fail >>> >> >>>>>>>>>>>> then it's deal breaking. >>> >> >>>>>>>>>>>> That is why we have added kerberos at the first place. >>> >> >>>>>>>>>>>> Unfortunately we >>> >> >>>>>>>>>>>> can't name them in this public list, however >>> >> >>>>>>>>>>>> the customers who specifically asked for this were >>> mainly in >>> >> >>>>>>>>>>>> the banking >>> >> >>>>>>>>>>>> and telco sector. >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> BR, >>> >> >>>>>>>>>>>> G >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> On Thu, Jun 3, 2021 at 9:20 AM Till Rohrmann < >>> >> >>>>>>>>>>>> trohrm...@apache.org> wrote: >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> > Thanks for updating the document Márton. Why is it that >>> >> banks >>> >> >>>>>>>>>>>> will >>> >> >>>>>>>>>>>> > consider it more secure if Flink comes with Kerberos >>> >> >>>>>>>>>>>> authentication >>> >> >>>>>>>>>>>> > (assuming a properly secured setup)? I mean if an >>> attacker >>> >> >>>>>>>>>>>> can get access >>> >> >>>>>>>>>>>> > to one of the machines, then it should also be >>> possible to >>> >> >>>>>>>>>>>> obtain the right >>> >> >>>>>>>>>>>> > Kerberos token. >>> >> >>>>>>>>>>>> > >>> >> >>>>>>>>>>>> > I am not an authentication expert and that's why I >>> wanted >>> >> to >>> >> >>>>>>>>>>>> ask what are >>> >> >>>>>>>>>>>> > other authentication protocols other than Kerberos? >>> Why did >>> >> >>>>>>>>>>>> we select >>> >> >>>>>>>>>>>> > Kerberos and not any other authentication protocol? >>> Maybe >>> >> you >>> >> >>>>>>>>>>>> can list the >>> >> >>>>>>>>>>>> > pros and cons for the different protocols. Is Kerberos >>> also >>> >> >>>>>>>>>>>> the standard >>> >> >>>>>>>>>>>> > authentication protocol for Kubernetes deployments? If >>> not, >>> >> >>>>>>>>>>>> what would be >>> >> >>>>>>>>>>>> > the answer when deploying on K8s? >>> >> >>>>>>>>>>>> > >>> >> >>>>>>>>>>>> > Cheers, >>> >> >>>>>>>>>>>> > Till >>> >> >>>>>>>>>>>> > >>> >> >>>>>>>>>>>> > On Wed, Jun 2, 2021 at 12:07 PM Gabor Somogyi < >>> >> >>>>>>>>>>>> gabor.g.somo...@gmail.com> >>> >> >>>>>>>>>>>> > wrote: >>> >> >>>>>>>>>>>> > >>> >> >>>>>>>>>>>> >> Hi team, >>> >> >>>>>>>>>>>> >> >>> >> >>>>>>>>>>>> >> Happy to be here and hope I can provide quality >>> additions >>> >> in >>> >> >>>>>>>>>>>> the future. >>> >> >>>>>>>>>>>> >> >>> >> >>>>>>>>>>>> >> Thank you all for helpful the suggestions! >>> >> >>>>>>>>>>>> >> Considering them the FLIP has been modified and the >>> work >>> >> >>>>>>>>>>>> continues on the >>> >> >>>>>>>>>>>> >> already existing Jira. >>> >> >>>>>>>>>>>> >> >>> >> >>>>>>>>>>>> >> BR, >>> >> >>>>>>>>>>>> >> G >>> >> >>>>>>>>>>>> >> >>> >> >>>>>>>>>>>> >> >>> >> >>>>>>>>>>>> >> On Wed, Jun 2, 2021 at 11:23 AM Márton Balassi < >>> >> >>>>>>>>>>>> balassi.mar...@gmail.com> >>> >> >>>>>>>>>>>> >> wrote: >>> >> >>>>>>>>>>>> >> >>> >> >>>>>>>>>>>> >>> Thanks, Chesney - I totally missed that. Answered on >>> the >>> >> >>>>>>>>>>>> ticket too, let >>> >> >>>>>>>>>>>> >>> us continue there then. >>> >> >>>>>>>>>>>> >>> >>> >> >>>>>>>>>>>> >>> Till, I agree that we should keep this codepath as >>> slim >>> >> as >>> >> >>>>>>>>>>>> possible. It >>> >> >>>>>>>>>>>> >>> is an important design decision that we aim to keep >>> the >>> >> >>>>>>>>>>>> list of >>> >> >>>>>>>>>>>> >>> authentication protocols to a minimum. We believe >>> that >>> >> this >>> >> >>>>>>>>>>>> should not be a >>> >> >>>>>>>>>>>> >>> primary concern of Flink and a trusted proxy service >>> (for >>> >> >>>>>>>>>>>> example Apache >>> >> >>>>>>>>>>>> >>> Knox) should be used to enable a multitude of enduser >>> >> >>>>>>>>>>>> authentication >>> >> >>>>>>>>>>>> >>> mechanisms. The bare minimum of authentication >>> mechanisms >>> >> >>>>>>>>>>>> to support >>> >> >>>>>>>>>>>> >>> consequently consist of a single strong >>> authentication >>> >> >>>>>>>>>>>> protocol for which >>> >> >>>>>>>>>>>> >>> Kerberos is the enterprise solution and HTTP Basic >>> >> primary >>> >> >>>>>>>>>>>> for development >>> >> >>>>>>>>>>>> >>> and light-weight scenarios. >>> >> >>>>>>>>>>>> >>> >>> >> >>>>>>>>>>>> >>> Added the above wording to G's doc. >>> >> >>>>>>>>>>>> >>> >>> >> >>>>>>>>>>>> >>> >>> >> >>>>>>>>>>>> >>> >> >>> https://docs.google.com/document/d/1NMPeJ9H0G49TGy3AzTVVJVKmYC0okwOtqLTSPnGqzHw/edit >>> >> >>>>>>>>>>>> >>> >>> >> >>>>>>>>>>>> >>> >>> >> >>>>>>>>>>>> >>> >>> >> >>>>>>>>>>>> >>> On Tue, Jun 1, 2021 at 11:47 AM Chesnay Schepler < >>> >> >>>>>>>>>>>> ches...@apache.org> >>> >> >>>>>>>>>>>> >>> wrote: >>> >> >>>>>>>>>>>> >>> >>> >> >>>>>>>>>>>> >>>> There's a related effort: >>> >> >>>>>>>>>>>> >>>> https://issues.apache.org/jira/browse/FLINK-21108 >>> >> >>>>>>>>>>>> >>>> >>> >> >>>>>>>>>>>> >>>> On 6/1/2021 10:14 AM, Till Rohrmann wrote: >>> >> >>>>>>>>>>>> >>>> > Hi Gabor, welcome to the Flink community! >>> >> >>>>>>>>>>>> >>>> > >>> >> >>>>>>>>>>>> >>>> > Thanks for sharing this proposal with the >>> community >>> >> >>>>>>>>>>>> Márton. In >>> >> >>>>>>>>>>>> >>>> general, I >>> >> >>>>>>>>>>>> >>>> > agree that authentication is missing and that >>> this is >>> >> >>>>>>>>>>>> required for >>> >> >>>>>>>>>>>> >>>> using >>> >> >>>>>>>>>>>> >>>> > Flink within an enterprise. The thing I am >>> wondering >>> >> is >>> >> >>>>>>>>>>>> whether this >>> >> >>>>>>>>>>>> >>>> > feature strictly needs to be implemented inside of >>> >> Flink >>> >> >>>>>>>>>>>> or whether a >>> >> >>>>>>>>>>>> >>>> proxy >>> >> >>>>>>>>>>>> >>>> > setup could do the job? Have you considered this >>> >> option? >>> >> >>>>>>>>>>>> If yes, then >>> >> >>>>>>>>>>>> >>>> it >>> >> >>>>>>>>>>>> >>>> > would be good to list it under the point of >>> rejected >>> >> >>>>>>>>>>>> alternatives. >>> >> >>>>>>>>>>>> >>>> > >>> >> >>>>>>>>>>>> >>>> > I do see the benefit of implementing this feature >>> >> inside >>> >> >>>>>>>>>>>> of Flink if >>> >> >>>>>>>>>>>> >>>> many >>> >> >>>>>>>>>>>> >>>> > users need it. If not, then it might be easier >>> for the >>> >> >>>>>>>>>>>> project to not >>> >> >>>>>>>>>>>> >>>> > increase the surface area since it makes the >>> overall >>> >> >>>>>>>>>>>> maintenance >>> >> >>>>>>>>>>>> >>>> harder. >>> >> >>>>>>>>>>>> >>>> > >>> >> >>>>>>>>>>>> >>>> > Cheers, >>> >> >>>>>>>>>>>> >>>> > Till >>> >> >>>>>>>>>>>> >>>> > >>> >> >>>>>>>>>>>> >>>> > On Mon, May 31, 2021 at 4:57 PM Márton Balassi < >>> >> >>>>>>>>>>>> mbala...@apache.org> >>> >> >>>>>>>>>>>> >>>> wrote: >>> >> >>>>>>>>>>>> >>>> > >>> >> >>>>>>>>>>>> >>>> >> Hi team, >>> >> >>>>>>>>>>>> >>>> >> >>> >> >>>>>>>>>>>> >>>> >> Firstly I would like to introduce Gabor or G [1] >>> for >>> >> >>>>>>>>>>>> short to the >>> >> >>>>>>>>>>>> >>>> >> community, he is a Spark committer who has >>> recently >>> >> >>>>>>>>>>>> transitioned to >>> >> >>>>>>>>>>>> >>>> the >>> >> >>>>>>>>>>>> >>>> >> Flink Engineering team at Cloudera and is looking >>> >> >>>>>>>>>>>> forward to >>> >> >>>>>>>>>>>> >>>> contributing >>> >> >>>>>>>>>>>> >>>> >> to Apache Flink. Previously G primarily focused >>> on >>> >> >>>>>>>>>>>> Spark Streaming >>> >> >>>>>>>>>>>> >>>> and >>> >> >>>>>>>>>>>> >>>> >> security. >>> >> >>>>>>>>>>>> >>>> >> >>> >> >>>>>>>>>>>> >>>> >> Based on requests from our customers G has >>> >> implemented >>> >> >>>>>>>>>>>> Kerberos and >>> >> >>>>>>>>>>>> >>>> HTTP >>> >> >>>>>>>>>>>> >>>> >> Basic Authentication for the Flink Dashboard and >>> >> >>>>>>>>>>>> HistoryServer. >>> >> >>>>>>>>>>>> >>>> Previously >>> >> >>>>>>>>>>>> >>>> >> lacked an authentication story. >>> >> >>>>>>>>>>>> >>>> >> >>> >> >>>>>>>>>>>> >>>> >> We are looking to contribute this functionality >>> back >>> >> to >>> >> >>>>>>>>>>>> the >>> >> >>>>>>>>>>>> >>>> community, we >>> >> >>>>>>>>>>>> >>>> >> believe that given Flink's maturity there should >>> be a >>> >> >>>>>>>>>>>> common code >>> >> >>>>>>>>>>>> >>>> solution >>> >> >>>>>>>>>>>> >>>> >> for this general pattern. >>> >> >>>>>>>>>>>> >>>> >> >>> >> >>>>>>>>>>>> >>>> >> We are looking forward to your feedback on G's >>> >> design. >>> >> >>>>>>>>>>>> [2] >>> >> >>>>>>>>>>>> >>>> >> >>> >> >>>>>>>>>>>> >>>> >> [1] http://gaborsomogyi.com/ >>> >> >>>>>>>>>>>> >>>> >> [2] >>> >> >>>>>>>>>>>> >>>> >> >>> >> >>>>>>>>>>>> >>>> >> >>> >> >>>>>>>>>>>> >>>> >>> >> >>>>>>>>>>>> >>> >> >>> https://docs.google.com/document/d/1NMPeJ9H0G49TGy3AzTVVJVKmYC0okwOtqLTSPnGqzHw/edit >>> >> >>>>>>>>>>>> >>>> >> >>> >> >>>>>>>>>>>> >>>> >>> >> >>>>>>>>>>>> >>>> >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>> >>> >> >>> > >>> > >>> > -- >>> > >>> > Konstantin Knauf >>> > >>> > https://twitter.com/snntrable >>> > >>> > https://github.com/knaufk >>> > >>> >> >> >> -- >> >> Konstantin Knauf >> >> https://twitter.com/snntrable >> >> https://github.com/knaufk >> >
