Re: [VOTE] FLIP-212: Introduce Flink Kubernetes Operator
Hi Thomas! +1 (binding) from my side Happy to see this effort getting some traction! Cheers, Gyula On Sat, Feb 5, 2022 at 3:00 AM Thomas Weise wrote: > Hi everyone, > > I'd like to start a vote on FLIP-212: Introduce Flink Kubernetes > Operator [1] which has been discussed in [2]. > > The vote will be open for at least 72 hours unless there is an > objection or not enough votes. > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-212%3A+Introduce+Flink+Kubernetes+Operator > [2] https://lists.apache.org/thread/1z78t6rf70h45v7fbd2m93rm2y1bvh0z > > Thanks! > Thomas >
Re: [DISCUSS] FLIP-212: Introduce Flink Kubernetes Operator
Hi everyone, just wanted to mention that my employer agreed to open source the PoC I developed: https://github.com/MicroFocus/opsb-flink-k8s-operator I understand the concern for maintainability, so Gradle & Kotlin might not be appealing to you, but at least it gives you another reference. The Helm resources in particular might be useful. There are bits and pieces there referring to Flink sessions, but those are just placeholders, the functioning parts use application mode with native integration. Regards, Alexis. From: Thomas Weise Sent: Saturday, February 5, 2022 2:41 AM To: dev Subject: Re: [DISCUSS] FLIP-212: Introduce Flink Kubernetes Operator Hi, Thanks for the continued feedback and discussion. Looks like we are ready to start a VOTE, I will initiate it shortly. In parallel it would be good to find the repository name. My suggestion would be: flink-kubernetes-operator I thought "flink-operator" could be a bit misleading since the term operator already has a meaning in Flink. I also considered "flink-k8s-operator" but that would be almost identical to existing operator implementations and could lead to confusion in the future. Thoughts? Thanks, Thomas On Fri, Feb 4, 2022 at 5:15 AM Gyula Fóra wrote: > > Hi Danny, > > So far we have been focusing our dev efforts on the initial native > implementation with the team. > If the discussion and vote goes well for this FLIP we are looking forward > to contributing the initial version sometime next week (fingers crossed). > > At that point I think we can already start the dev work to support the > standalone mode as well, especially if you can dedicate some effort to > pushing that side. > Working together on this sounds like a great idea and we should start as > soon as possible! :) > > Cheers, > Gyula > > On Fri, Feb 4, 2022 at 2:07 PM Danny Cranmer > wrote: > > > I have been discussing this one with my team. We are interested in the > > Standalone mode, and are willing to contribute towards the implementation. > > Potentially we can work together to support both modes in parallel? > > > > Thanks, > > > > On Wed, Feb 2, 2022 at 4:02 PM Gyula Fóra wrote: > > > > > Hi Danny! > > > > > > Thanks for the feedback :) > > > > > > Versioning: > > > Versioning will be independent from Flink and the operator will depend > > on a > > > fixed flink version (in every given operator version). > > > This should be the exact same setup as with Stateful Functions ( > > > https://github.com/apache/flink-statefun). So independent release cycle > > > but > > > still within the Flink umbrella. > > > > > > Deployment error handling: > > > I think that's a very good point, as general exception handling for the > > > different failure scenarios is a tricky problem. I think the exception > > > classifiers and retry strategies could avoid a lot of manual intervention > > > from the user. We will definitely need to add something like this. Once > > we > > > have the repo created with the initial operator code we should open some > > > tickets for this and put it on the short term roadmap! > > > > > > Cheers, > > > Gyula > > > > > > On Wed, Feb 2, 2022 at 4:50 PM Danny Cranmer > > > wrote: > > > > > > > Hey team, > > > > > > > > Great work on the FLIP, I am looking forward to this one. I agree that > > we > > > > can move forward to the voting stage. > > > > > > > > I have general feedback around how we will handle job submission > > failure > > > > and retry. As discussed in the Rejected Alternatives section, we can > > use > > > > Java to handle job submission failures from the Flink client. It would > > be > > > > useful to have the ability to configure exception classifiers and retry > > > > strategy as part of operator configuration. > > > > > > > > Given this will be in a separate Github repository I am curious how > > ther > > > > versioning strategy will work in relation to the Flink version? Do we > > > have > > > > any other components with a similar setup I can look at? Will the > > > operator > > > > version track Flink or will it use its own versioning strategy with a > > > Flink > > > > version support matrix, or similar? > > > > > > > > Thanks, > > > > > > > > > > > > > > > > On Tue, Feb 1, 2022 at 2:33 PM Márton Balassi < > > balassi.mar...@gmail.com> > > > > wrote: > > > > > > > > > Hi team, > > > > > > > > > > Thank you for the great feedback, Thomas has updated the FLIP page > > > > > accordingly. If you are comfortable with the currently existing > > design > > > > and > > > > > depth in the FLIP [1] I suggest moving forward to the voting stage - > > > once > > > > > that reaches a positive conclusion it lets us create the separate > > code > > > > > repository under the flink project for the operator. > > > > > > > > > > I encourage everyone to keep improving the details in the meantime, > > > > however > > > > > I believe given the existing design and the general sentiment on this > > > > > thread that t
Re: [VOTE] FLIP-212: Introduce Flink Kubernetes Operator
+1 (non-binding) Thanks folks for leading this effort and making it happen so fast! Best, Chenya On Sat, Feb 5, 2022 at 12:02 AM Gyula Fóra wrote: > Hi Thomas! > > +1 (binding) from my side > > Happy to see this effort getting some traction! > > Cheers, > Gyula > > On Sat, Feb 5, 2022 at 3:00 AM Thomas Weise wrote: > > > Hi everyone, > > > > I'd like to start a vote on FLIP-212: Introduce Flink Kubernetes > > Operator [1] which has been discussed in [2]. > > > > The vote will be open for at least 72 hours unless there is an > > objection or not enough votes. > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-212%3A+Introduce+Flink+Kubernetes+Operator > > [2] https://lists.apache.org/thread/1z78t6rf70h45v7fbd2m93rm2y1bvh0z > > > > Thanks! > > Thomas > > >
Re: [VOTE] FLIP-212: Introduce Flink Kubernetes Operator
+1 (non-binding) On Sat, Feb 5, 2022 at 4:39 PM Chenya Zhang wrote: > +1 (non-binding) > > Thanks folks for leading this effort and making it happen so fast! > > Best, > Chenya > > On Sat, Feb 5, 2022 at 12:02 AM Gyula Fóra wrote: > > > Hi Thomas! > > > > +1 (binding) from my side > > > > Happy to see this effort getting some traction! > > > > Cheers, > > Gyula > > > > On Sat, Feb 5, 2022 at 3:00 AM Thomas Weise wrote: > > > > > Hi everyone, > > > > > > I'd like to start a vote on FLIP-212: Introduce Flink Kubernetes > > > Operator [1] which has been discussed in [2]. > > > > > > The vote will be open for at least 72 hours unless there is an > > > objection or not enough votes. > > > > > > [1] > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-212%3A+Introduce+Flink+Kubernetes+Operator > > > [2] https://lists.apache.org/thread/1z78t6rf70h45v7fbd2m93rm2y1bvh0z > > > > > > Thanks! > > > Thomas > > > > > >
Re: [VOTE] FLIP-212: Introduce Flink Kubernetes Operator
I am very excited to see this. Thanks for driving the effort +1 (non-binding) On Sat, Feb 5, 2022 at 10:53 AM Shqiprim Bunjaku wrote: > +1 (non-binding) > > > > On Sat, Feb 5, 2022 at 4:39 PM Chenya Zhang > wrote: > > > +1 (non-binding) > > > > Thanks folks for leading this effort and making it happen so fast! > > > > Best, > > Chenya > > > > On Sat, Feb 5, 2022 at 12:02 AM Gyula Fóra wrote: > > > > > Hi Thomas! > > > > > > +1 (binding) from my side > > > > > > Happy to see this effort getting some traction! > > > > > > Cheers, > > > Gyula > > > > > > On Sat, Feb 5, 2022 at 3:00 AM Thomas Weise wrote: > > > > > > > Hi everyone, > > > > > > > > I'd like to start a vote on FLIP-212: Introduce Flink Kubernetes > > > > Operator [1] which has been discussed in [2]. > > > > > > > > The vote will be open for at least 72 hours unless there is an > > > > objection or not enough votes. > > > > > > > > [1] > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-212%3A+Introduce+Flink+Kubernetes+Operator > > > > [2] https://lists.apache.org/thread/1z78t6rf70h45v7fbd2m93rm2y1bvh0z > > > > > > > > Thanks! > > > > Thomas > > > > > > > > > > -- Israel Ekpo Lead Instructor, IzzyAcademy.com https://www.youtube.com/c/izzyacademy https://izzyacademy.com/
[VOTE] Release 1.13.6, release candidate #1
Hi everyone, Please review and vote on the release candidate #1 for the version 1.13.6, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1], * the official Apache source release and binary convenience releases to be deployed to dist.apache.org [2], which are signed with the key with fingerprint 8C3FB007FE60 DEFA [3], * all artifacts to be deployed to the Maven Central Repository [4], * source code tag "release-1.2.3-rc3" [5], * website pull request listing the new release and adding announcement blog post [6]. The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. Thanks, Konstantin [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12351074 [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.13.6-rc1/ [3] https://dist.apache.org/repos/dist/release/flink/KEYS [4] https://repository.apache.org/content/repositories/orgapacheflink-1486/ [5] https://github.com/apache/flink/tree/release-1.13.6-rc1 [6] https://github.com/apache/flink-web/pull/505 -- Konstantin Knauf https://twitter.com/snntrable https://github.com/knaufk
Re: [VOTE] Deprecate Per-Job Mode in Flink 1.15
Thanks Konstantin for the explanation. +1 (binding) from me now. Best, Yang Xintong Song 于2022年2月3日周四 09:42写道： > Thanks for the clarification, Konstantin. > > +1 for deprecating per-job mode in Flink 1.15, and reevaluating when to > drop it after Flink 1.16. > > Thank you~ > > Xintong Song > > > > On Tue, Feb 1, 2022 at 5:27 PM Konstantin Knauf wrote: > > > Hi Xintong, Hi Yang, Hi everyone, > > > > Thank you for speaking up. The vote is formally only about the > deprecation > > in Flink 1.15. > > > > We can and should continue to collect blockers for the deletion of > per-job > > mode on YARN. Then there should be one release that allows users to > switch. > > So, Flink 1.16 indeed is unrealistic for dropping, as we would need to > > address all Blockers still in Flink 1.15. > > > > I think a certain degree of urgency helps us to address these issues and > > encourages users to switch to application mode. So, I would continue to > > target Flink 1.17 for dropping per-job mode, but let's reevaluate after > > Flink 1.16. > > > > Hope this helps, > > > > Konstantin > > > > Since we recently decided that > > On Sun, Jan 30, 2022 at 4:13 AM Yang Wang wrote: > > > > > Hi all, > > > > > > > > > > > > I second Xintong’s comments to not drop the per-job mode too > > aggressively. > > > And I am afraid > > > > > > we need to get more inputs from users after deprecating the per-job > mode > > in > > > release-1.15. > > > > > > > > > Most Flink on YARN users are using CLI command to integrate with the > job > > > lifecycle management system. > > > > > > And they are still using the old compatibility mode "flink run -m > > > yarn-cluster", not the generic CLI mode "--target > > > yarn-per-job/yarn-application". > > > > > > Apart from the functionalities, they need some time to upgrade the > > external > > > systems. > > > > > > > > > BTW, the application mode does not support attached mode now. Some > users > > > have asked for this in FLINK-25495[1]. > > > > > > > > > [1]. https://issues.apache.org/jira/browse/FLINK-25495 > > > > > > > > > > > > > > > Best, > > > > > > Yang > > > > > > Xintong Song 于2022年1月30日周日 08:35写道： > > > > > > > Hi Konstantin, > > > > > > > > Could we be more specific about what this vote is for? I'm asking > > > because I > > > > don't think we have consensus on all you have mentioned. > > > > > > > > To be specific, I'd be +1 for deprecating per-job mode in 1.15. > > However, > > > > I'm not sure about the following. > > > > - Targeting to drop it in 1.16 or 1.17. TBH, I'd expect to stay > > > compatible > > > > on the per-job mode a bit longer. > > > > - Targeting Yarn application mode on par with the standalone / K8s. I > > > think > > > > we need the Yarn application mode on par with the Yarn per-job mode, > as > > > the > > > > latter is being dropped and users are migrating from. > > > > - FLINK-24897 being the only blocker for dropping the per-job mode. I > > > think > > > > a good time to drop the per-job mode is probably when we know most > > users > > > > have migrated to the application mode. Even if the Yarn application > > mode > > > > provides equivalent functionality as the Yarn per-job mode does, it's > > > > probably nicer to not force users to migrate if the per-job mode is > > still > > > > widely used. > > > > > > > > Discussing the above items is not my purpose here. Just trying to say > > > that > > > > IMHO in the previous discussion [1] we have not reached consensus on > > all > > > > the things mentioned in this voting thread. Consequently, if these > are > > > all > > > > included in the scope of the vote, I'm afraid I cannot give my +1 on > > > this. > > > > Sorry if I'm nitpicking. > > > > > > > > Thank you~ > > > > > > > > Xintong Song > > > > > > > > > > > > [1] https://lists.apache.org/thread/b8g76cqgtr2c515rd1bs41vy285f317n > > > > > > > > On Sat, Jan 29, 2022 at 2:27 PM Jing Zhang > > wrote: > > > > > > > > > +1 (binding) > > > > > > > > > > Thanks Konstantin for driving this. > > > > > > > > > > Best, > > > > > Jing Zhang > > > > > > > > > > Chenya Zhang 于2022年1月29日周六 07:04写道： > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > On Fri, Jan 28, 2022 at 12:46 PM Thomas Weise > > > wrote: > > > > > > > > > > > > > +1 (binding) > > > > > > > > > > > > > > On Fri, Jan 28, 2022 at 9:27 AM David Morávek > > > > > wrote: > > > > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > > > > > D. > > > > > > > > > > > > > > > > On Fri 28. 1. 2022 at 17:53, Till Rohrmann < > > trohrm...@apache.org > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > +1 (binding) > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > Till > > > > > > > > > > > > > > > > > > On Fri, Jan 28, 2022 at 4:57 PM Gabor Somogyi < > > > > > > > gabor.g.somo...@gmail.com > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > > > > > > > > > We're intended to make tests when FLINK-24897 > >
