Hi,
Thanks for the quick response! I think this looks good now and it should be
something that everyone can agree on as a first step.
Best,
Aljoscha
> On 6. Sep 2019, at 12:22, Dian Fu wrote:
>
> Hi all,
>
> I have updated the FLIP and removed content relate to UDAF and also changed
> the t
Hi all,
I have updated the FLIP and removed content relate to UDAF and also changed the
title of the FLIP to "Flink Python User-Defined Stateless Function for Table".
Does it make sense to you?
Regards,
Dian
> 在 2019年9月6日，下午6:09，Dian Fu 写道：
>
> Hi all,
>
> Thanks a lot for the discussion h
Hi all,
Thanks a lot for the discussion here. It makes sense to limit the scope of this
FLIP to only ScalarFunction. I'll update the FLIP and remove the content
relating to UDAF.
Thanks,
Dian
> 在 2019年9月6日，下午6:02，jincheng sun 写道：
>
> Hi，
>
> Sure, for ensure the 1.10 relesae of flink, let's
Hi，
Sure, for ensure the 1.10 relesae of flink, let's split the FLIPs, and
FLIP-58 only do the stateless part.
Cheers,
Jincheng
Aljoscha Krettek 于2019年9月6日周五 下午5:53写道：
> Hi,
>
> Regarding stateful functions and MapView/DataView/ListView: I think it’s
> best to keep that for a later FLIP and fo
Hi,
Regarding stateful functions and MapView/DataView/ListView: I think it’s best
to keep that for a later FLIP and focus on a more basic version. Supporting
stateful functions, especially with MapView can potentially be very slow so we
have to see what we can do there.
For the method names, I
Hi Aljoscha,
Thanks for your comments!
Regarding to the FLIP scope, it seems that we have agreed on the design of
the stateless function support.
What do you think about starting the development of the stateless function
support firstly and continue the discussion of stateful function support?
Or
Hi,
Another thing to consider is the Scope of the FLIP. Currently, we try to
support (stateful) AggregateFunctions. I have some concerns about whether or
not DataView/MapView/ListView is a good interface because it requires quite
some magic from the runners to make it work, such as messing with
Hi Aljoscha,
That's a good points, so far, most of the code will live in flink-python
module, and the rules and relNodes will be put into the both blink and
flink planner modules, some of the common interface of required by planners
will be placed in flink-table-common. I think you are right, we s
Hi Timo,
Yes, I think about convenient is a trade-off, for now think convenient
compare with always do right thing I prefer add some limitation ensure user
never encounter issues. So, #2 we are on the same page now.
Best,
Jincheng
Aljoscha Krettek 于2019年9月4日周三 下午8:58写道：
> Hi,
>
> Things looks
Hi,
Things looks interesting so far!
I had one question: Where will most of the support code for this live? Will
this add the required code to flink-table-common or the different runners? Can
we implement this in such a way that only a minimal amount of support code is
required in the parts of
Hi Jincheng,
2. Serializability of functions: "#2 is very convenient for users" means
only until they have the first backwards-compatibility issue, after that
they will find it not so convinient anymore and will ask why the
framework allowed storing such objects in a persistent storage. I don'
Hi Timo,
Thanks for the quick reply ! :)
I have added more example for #3 and #5 to the FLIP. That are great
suggestions !
Regarding 2:
There are two kind Serialization for CloudPickle(Which is different from
Java):
1) For class and function which can be imported, CloudPickle only
serialize the
Hi Jincheng,
thanks for your response.
2. Serializability of functions: Using some arbitrary serialization
format for shipping a function to worker sounds fine to me. But once we
store functions a the catalog we need to think about backwards
compatibility and evolution of interfaces etc. I'm
Hi Timo,
Great thanks for your feedback. I would like to share my thoughts with you
inline. :)
Best,
Jincheng
Timo Walther 于2019年9月2日周一 下午5:04写道：
> Hi all,
>
> the FLIP looks awesome. However, I would like to discuss the changes to
> the user-facing parts again. Some feedback:
>
> 1. DataViews
Hi Shaoxuan,
Thanks for reminding that. I think "Flink Python User-Defined Function for
Table" make sense to me.
Best,
Jincheng
Timo Walther 于2019年9月2日周一 下午5:04写道：
> Hi all,
>
> the FLIP looks awesome. However, I would like to discuss the changes to
> the user-facing parts again. Some feedbac
Hi all,
the FLIP looks awesome. However, I would like to discuss the changes to
the user-facing parts again. Some feedback:
1. DataViews: With the current non-annotation design for DataViews, we
cannot perform eager state declaration, right? At which point during
execution do we know which s
Hi Jincheng, Fudian, and Aljoscha,
I am assuming the proposed python UDX can also be applied to Flink SQL.
Is this correct? If yes, I would suggest to title the FLIP as "Flink Python
User-Defined Function" or "Flink Python User-Defined Function for Table".
Regards,
Shaoxuan
On Wed, Aug 28, 2019
Thanks for the feedback Bowen!
Great thanks for create the FLIP and bring up the VOTE Dian!
Best, Jincheng
Dian Fu 于2019年8月28日周三 上午11:32写道：
> Hi all,
>
> I have started a voting thread [1]. Thanks a lot for your help during
> creating the FLIP @Jincheng.
>
>
> Hi Bowen,
>
> Very appreciated fo
Hi all,
I have started a voting thread [1]. Thanks a lot for your help during creating
the FLIP @Jincheng.
Hi Bowen,
Very appreciated for your comments. I have replied you in the design doc. As it
seems that the comments doesn't affect the overall design, I'll not cancel the
vote for now and
Hi Jincheng and Dian,
Sorry for being late to the party. I took a glance at the proposal, LGTM in
general, and I left only a couple comments.
Thanks,
Bowen
On Mon, Aug 26, 2019 at 8:05 PM Dian Fu wrote:
> Hi Jincheng,
>
> Thanks! It works.
>
> Thanks,
> Dian
>
> > 在 2019年8月27日，上午10:55，jinchen
Hi Jincheng,
Thanks! It works.
Thanks,
Dian
> 在 2019年8月27日，上午10:55，jincheng sun 写道：
>
> Hi Dian, can you check if you have edit access? :)
>
>
> Dian Fu 于2019年8月26日周一 上午10:52写道：
>
>> Hi Jincheng,
>>
>> Appreciated for the kind tips and offering of help. Definitely need it!
>> Could you gr
Hi Dian, can you check if you have edit access? :)
Dian Fu 于2019年8月26日周一 上午10:52写道：
> Hi Jincheng,
>
> Appreciated for the kind tips and offering of help. Definitely need it!
> Could you grant me write permission for confluence? My Id: Dian Fu
>
> Thanks,
> Dian
>
> > 在 2019年8月26日，上午9:53，jinche
Hi Jincheng,
Appreciated for the kind tips and offering of help. Definitely need it! Could
you grant me write permission for confluence? My Id: Dian Fu
Thanks,
Dian
> 在 2019年8月26日，上午9:53，jincheng sun 写道：
>
> Thanks for your feedback Hequn & Dian.
>
> Dian, I am glad to see that you want help
Thanks for your feedback Hequn & Dian.
Dian, I am glad to see that you want help to create the FLIP!
Everyone will have first time, and I am very willing to help you complete
your first FLIP creation. Here some tips:
- First I'll give your account write permission for confluence.
- Before create
+1 for starting the vote.
Thanks Jincheng a lot for the discussion.
Best, Hequn
On Fri, Aug 23, 2019 at 10:06 AM Dian Fu wrote:
> Hi Jincheng,
>
> +1 to start the FLIP create and VOTE on this feature. I'm willing to help
> on the FLIP create if you don't mind. As I haven't created a FLIP befor
Hi Jincheng,
+1 to start the FLIP create and VOTE on this feature. I'm willing to help on
the FLIP create if you don't mind. As I haven't created a FLIP before, it will
be great if you could help on this. :)
Regards,
Dian
> 在 2019年8月22日，下午11:41，jincheng sun 写道：
>
> Hi all,
>
> Thanks a lot
Hi all,
Thanks a lot for your feedback. If there are no more suggestions and
comments, I think it's better to initiate a vote to create a FLIP for
Apache Flink Python UDFs.
What do you think?
Best, Jincheng
jincheng sun 于2019年8月15日周四 上午12:54写道：
> Hi Thomas,
>
> Thanks for your confirmation an
Hi Thomas,
Thanks for your confirmation and the very important reminder about bundle
processing.
I have had add the description about how to perform bundle processing from
the perspective of checkpoint and watermark. Feel free to leave comments if
there are anything not describe clearly.
Best,
J
Hi Thomas,
Thanks a lot the suggestions.
Regarding to bundle processing, there is a section "Checkpoint"[1] in the
design doc which talks about how to handle the checkpoint.
However, I think you are right that we should talk more about it, such as
what's bundle processing, how it affects the ch
Hi Jincheng,
Thanks for putting this together. The proposal is very detailed, thorough
and for me as a Beam Flink runner contributor easy to understand :)
One thing that you should probably detail more is the bundle processing. It
is critically important for performance that multiple elements are
30 matches
Mail list logo
