Thanks for your feedback Hequn & Dian. Dian, I am glad to see that you want help to create the FLIP! Everyone will have first time, and I am very willing to help you complete your first FLIP creation. Here some tips:
- First I'll give your account write permission for confluence. - Before create the FLIP, please have look at the FLIP Template [1], (It's better to know more about FLIP by reading [2]) - Create Flink Python UDFs related JIRAs after completing the VOTE of FLIP.(I think you also can bring up the VOTE thread, if you want! ) Any problems you encounter during this period，feel free to tell me that we can solve them together. :) Best, Jincheng [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP+Template [2] https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals Hequn Cheng <chenghe...@gmail.com> 于2019年8月23日周五 上午11:54写道： > +1 for starting the vote. > > Thanks Jincheng a lot for the discussion. > > Best, Hequn > > On Fri, Aug 23, 2019 at 10:06 AM Dian Fu <dian0511...@gmail.com> wrote: > > > Hi Jincheng, > > > > +1 to start the FLIP create and VOTE on this feature. I'm willing to help > > on the FLIP create if you don't mind. As I haven't created a FLIP before, > > it will be great if you could help on this. :) > > > > Regards, > > Dian > > > > > 在 2019年8月22日，下午11:41，jincheng sun <sunjincheng...@gmail.com> 写道： > > > > > > Hi all, > > > > > > Thanks a lot for your feedback. If there are no more suggestions and > > > comments, I think it's better to initiate a vote to create a FLIP for > > > Apache Flink Python UDFs. > > > What do you think? > > > > > > Best, Jincheng > > > > > > jincheng sun <sunjincheng...@gmail.com> 于2019年8月15日周四 上午12:54写道： > > > > > >> Hi Thomas, > > >> > > >> Thanks for your confirmation and the very important reminder about > > bundle > > >> processing. > > >> > > >> I have had add the description about how to perform bundle processing > > from > > >> the perspective of checkpoint and watermark. Feel free to leave > > comments if > > >> there are anything not describe clearly. > > >> > > >> Best, > > >> Jincheng > > >> > > >> > > >> Dian Fu <dian0511...@gmail.com> 于2019年8月14日周三 上午10:08写道： > > >> > > >>> Hi Thomas, > > >>> > > >>> Thanks a lot the suggestions. > > >>> > > >>> Regarding to bundle processing, there is a section "Checkpoint"[1] in > > the > > >>> design doc which talks about how to handle the checkpoint. > > >>> However, I think you are right that we should talk more about it, > such > > as > > >>> what's bundle processing, how it affects the checkpoint and > watermark, > > how > > >>> to handle the checkpoint and watermark, etc. > > >>> > > >>> [1] > > >>> > > > https://docs.google.com/document/d/1WpTyCXAQh8Jr2yWfz7MWCD2-lou05QaQFb810ZvTefY/edit#heading=h.urladt565yo3 > > >>> < > > >>> > > > https://docs.google.com/document/d/1WpTyCXAQh8Jr2yWfz7MWCD2-lou05QaQFb810ZvTefY/edit#heading=h.urladt565yo3 > > >>>> > > >>> > > >>> Regards, > > >>> Dian > > >>> > > >>>> 在 2019年8月14日，上午1:01，Thomas Weise <t...@apache.org> 写道： > > >>>> > > >>>> Hi Jincheng, > > >>>> > > >>>> Thanks for putting this together. The proposal is very detailed, > > >>> thorough > > >>>> and for me as a Beam Flink runner contributor easy to understand :) > > >>>> > > >>>> One thing that you should probably detail more is the bundle > > >>> processing. It > > >>>> is critically important for performance that multiple elements are > > >>>> processed in a bundle. The default bundle size in the Flink runner > is > > >>> 1s or > > >>>> 1000 elements, whichever comes first. And for streaming, you can > find > > >>> the > > >>>> logic necessary to align the bundle processing with watermarks and > > >>>> checkpointing here: > > >>>> > > >>> > > > https://github.com/apache/beam/blob/release-2.14.0/runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/ExecutableStageDoFnOperator.java > > >>>> > > >>>> Thomas > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> On Tue, Aug 13, 2019 at 7:05 AM jincheng sun < > > sunjincheng...@gmail.com> > > >>>> wrote: > > >>>> > > >>>>> Hi all, > > >>>>> > > >>>>> The Python Table API(without Python UDF support) has already been > > >>> supported > > >>>>> and will be available in the coming release 1.9. > > >>>>> As Python UDF is very important for Python users, we'd like to > start > > >>> the > > >>>>> discussion about the Python UDF support in the Python Table API. > > >>>>> Aljoscha Krettek, Dian Fu and I have discussed offline and have > > >>> drafted a > > >>>>> design doc[1]. It includes the following items: > > >>>>> > > >>>>> - The user-defined function interfaces. > > >>>>> - The user-defined function execution architecture. > > >>>>> > > >>>>> As mentioned by many guys in the previous discussion thread[2], a > > >>>>> portability framework was introduced in Apache Beam in latest > > >>> releases. It > > >>>>> provides well-defined, language-neutral data structures and > protocols > > >>> for > > >>>>> language-neutral user-defined function execution. This design is > > based > > >>> on > > >>>>> Beam's portability framework. We will introduce how to make use of > > >>> Beam's > > >>>>> portability framework for user-defined function execution: data > > >>>>> transmission, state access, checkpoint, metrics, logging, etc. > > >>>>> > > >>>>> Considering that the design relies on Beam's portability framework > > for > > >>>>> Python user-defined function execution and not all the contributors > > in > > >>>>> Flink community are familiar with Beam's portability framework, we > > have > > >>>>> done a prototype[3] for proof of concept and also ease of > > >>> understanding of > > >>>>> the design. > > >>>>> > > >>>>> Welcome any feedback. > > >>>>> > > >>>>> Best, > > >>>>> Jincheng > > >>>>> > > >>>>> [1] > > >>>>> > > >>>>> > > >>> > > > https://docs.google.com/document/d/1WpTyCXAQh8Jr2yWfz7MWCD2-lou05QaQFb810ZvTefY/edit?usp=sharing > > >>>>> [2] > > >>>>> > > >>>>> > > >>> > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-38-Support-python-language-in-flink-TableAPI-td28061.html > > >>>>> [3] https://github.com/dianfu/flink/commits/udf_poc > > >>>>> > > >>> > > >>> > > > > >
