Re: Python Job Support for the Kubernetes Operator

2022-07-06 Thread Yang Wang
I think maybe we make the SQL or Python jobs submission more convenient in the future by introducing the *"**pyScriptURL"* or *"SQLScriptURI"* similar to *"jarURI"*. Best, Yang Thomas Weise 于2022年7月7日周四 01:06写道: > Since SQL or Python are essentially just examples of how to use the > operator v

Re: Python Job Support for the Kubernetes Operator

2022-07-06 Thread Thomas Weise
Since SQL or Python are essentially just examples of how to use the operator vs. features of the operator itself, they should not affect the release schedule and can be added anytime, as examples to the operator or elsewhere. Thanks, Thomas On Wed, Jul 6, 2022 at 8:33 AM Gyula Fóra wrote: > > Hi

Re: Python Job Support for the Kubernetes Operator

2022-07-06 Thread Gyula Fóra
Hi All! One thing we could do already now is to add a simple example on how to execute Python jobs like java jobs (with the right main class, args etc). It would be similar to https://github.com/apache/flink-kubernetes-operator/tree/main/examples/flink-sql-runner-example but slightly simpler as w

Re: Python Job Support for the Kubernetes Operator

2022-07-05 Thread Dian Fu
Thanks for the confirmation Matyas! On Tue, Jul 5, 2022 at 3:00 PM Őrhidi Mátyás wrote: > Yes, this is the plan Dian. Appreciate your assistance! > > Best, > Matyas > > On Tue, Jul 5, 2022 at 8:55 AM Dian Fu wrote: > >> Hi Matyas, >> >> According to the release schedule defined in [1], it seems

Re: Python Job Support for the Kubernetes Operator

2022-07-05 Thread Őrhidi Mátyás
Yes, this is the plan Dian. Appreciate your assistance! Best, Matyas On Tue, Jul 5, 2022 at 8:55 AM Dian Fu wrote: > Hi Matyas, > > According to the release schedule defined in [1], it seems that the > feature freeze of v1.2 may occur at the beginning of September, is this > correct? If this is

Re: Python Job Support for the Kubernetes Operator

2022-07-04 Thread Dian Fu
Hi Matyas, According to the release schedule defined in [1], it seems that the feature freeze of v1.2 may occur at the beginning of September, is this correct? If this is the case, I think it should be reasonable to make it in v1.2 for Python support. Regards, Dian [1] https://cwiki.apache.org/c

Re: Python Job Support for the Kubernetes Operator

2022-07-04 Thread Őrhidi Mátyás
Both sql and py support is requested frequently. I guess we should aim to support both in v1.2. Matyas On Tue, Jul 5, 2022 at 6:26 AM Gyula Fóra wrote: > Thank you for the info and help Dian :) > > Gyula > > On Tue, 5 Jul 2022 at 05:14, Yang Wang wrote: > > > Thanks Dian for the confirmation a

Re: Python Job Support for the Kubernetes Operator

2022-07-04 Thread Gyula Fóra
Thank you for the info and help Dian :) Gyula On Tue, 5 Jul 2022 at 05:14, Yang Wang wrote: > Thanks Dian for the confirmation and nice help. > > Best, > Yang > > Dian Fu 于2022年7月5日周二 09:27写道: > > > @Yang, Yes, you are right. Python jobs could be seen as special JAR jobs > > whose main class i

Re: Python Job Support for the Kubernetes Operator

2022-07-04 Thread Yang Wang
Thanks Dian for the confirmation and nice help. Best, Yang Dian Fu 于2022年7月5日周二 09:27写道: > @Yang, Yes, you are right. Python jobs could be seen as special JAR jobs > whose main class is always `org.apache.flink.client.python.PythonDriver`. > What we could do in Flink K8s operator is to make it

Re: Python Job Support for the Kubernetes Operator

2022-07-04 Thread Dian Fu
@Yang, Yes, you are right. Python jobs could be seen as special JAR jobs whose main class is always `org.apache.flink.client.python.PythonDriver`. What we could do in Flink K8s operator is to make it more convenient and handle properly for the different kinds of dependencies[1]. @Gyula, I can help

Re: Python Job Support for the Kubernetes Operator

2022-07-03 Thread Yang Wang
AFAIK, the python job could be considered as a special case of jar job. The user jar is flink-python-*.jar and is located in the opt directory. The python script is just the argument of this user jar. So I believe the users already could submit python jobs via Flink Kubernetes operator. However, th

Python Job Support for the Kubernetes Operator

2022-07-03 Thread Gyula Fóra
Hi Devs! Would anyone with a good understanding of the Python execution layer be interested in working on adding Python job support for the Flink Kubernetes Operator? This is a feature request that comes up often ( https://issues.apache.org/jira/browse/FLINK-28364) and it would be a great way to