Hey Gabor,
These are some meaningful concerns!
Joining data outside one's scope is somewhat similar to a “goto” statement when
you write imperative code. It certainly allows you to have more complex data
flow but then...programs become arbitrary (nested structures have no meaning).
On top of t
Hi Paris and Fouad,
I finally had some time to delve into this. Thanks for the nice proposal!
+1 for also having a CoLoopFunction. That might be useful even if the
input and feedback have the same type, as it might happen that I want
to treat the elements coming on the feedback in a different way
Hi again folks,
It has been almost a month so I would like to remind you about the loop
redesign which many of you have checked already.
Anyone else excited about iterative streams and wants to add a comment or
simply +1 this? It would be more than welcome :)
You can find the description in the
That was fast! Seems to be working.
Thank you Fabian!
> On 17 Nov 2016, at 13:58, Fabian Hueske wrote:
>
> Hi Paris,
>
> just gave you the permissions (I hope).
> Let me know if something does not work.
>
> Cheers, Fabian
>
> 2016-11-17 13:48 GMT+01:00 Paris Carbone :
>
>> We do not have
Hi Paris,
just gave you the permissions (I hope).
Let me know if something does not work.
Cheers, Fabian
2016-11-17 13:48 GMT+01:00 Paris Carbone :
> We do not have to schedule this for an early Flink release, just saying.
> I would just like to get the changes out and you people can review it
We do not have to schedule this for an early Flink release, just saying.
I would just like to get the changes out and you people can review it and
integrate it anytime at your own pace.
Who is the admin of the wiki? It would be nice to get write access.
> On 17 Nov 2016, at 13:45, Paris Carbone
Sounds like a plan!
Can someone grant me access to write in the wiki please?
My username is “senorcarbone”.
Paris
> On 16 Nov 2016, at 14:30, Gyula Fóra wrote:
>
> I am not completely sure whether we should deprecate the old API for 1.2 or
> remove it completely. Personally I am in favor of re
I am not completely sure whether we should deprecate the old API for 1.2 or
remove it completely. Personally I am in favor of removing it, I don't
think it is a huge burden to move to the new one if it makes for a much
nicer user experience.
I think you can go ahead add the FLIP to the wiki and op
Thanks for reviewing, Gyula.
One thing that is still up to discussion is whether we should remove completely
the old iterations API or simply mark it as deprecated till v2.0.
Also, not sure what is the best process now. We have the changes ready. Should
I copy the FLIP to the wiki and trigger th
Hi Paris,
I like the proposed changes to the iteration API, this cleans up things in
the Java API without any strict restriction I think (it was never a problem
in the Scala API).
The termination algorithm based on the proposed scoped loops seems to be
fairly simple and looks good :)
Cheers,
Gyu
That would be great Shi! Let's take that offline.
Anyone else interested in the iteration changes? It would be nice to
incorporate these to v1.2 if possible so I count on your review asap.
cheers,
Paris
On Nov 14, 2016, at 2:59 AM, xiaogang.sxg
mailto:xiaogang@alibaba-inc.com>> wrote:
Hi
Hi Shi,
Naiad/Timely Dataflow and other projects use global coordination which is very
convenient for asynchronous progress tracking in general but it has some
downsides in a production systems that count on in-flight transactional control
mechanisms and rollback recovery guarantees. This is wh
Hi, Fouad
Thank you for the explanation. Now the centralized method seems correct to
me.
The passing of StatusUpdate events will lead to synchronous iterations and
we are using the information in each iterations to terminate the
computation.
Actually, i prefer the centralized method because in ma
Hi Shi,
It seems that you are referring to the centralized algorithm which is no longer
the proposed version.
In the decentralized version (check last doc) there is no master node or global
coordination involved.
Let us keep this discussion to the decentralized one if possible.
To answer your
Hi Paris
I have several concerns about the correctness of the termination protocol.
I think the termination protocol put an end to the computation even when
the computation has not converged.
Suppose there exists a loop context constructed by a OP operator, a Head
operator and a Tail operator (il
Hi again Flink folks,
Here is our new proposal that addresses Job Termination - the loop fault
tolerance proposal will follow shortly.
As Stephan hinted, we need operators to be aware of their scope level.
Thus, it is time we make loops great again! :)
Part of this FLIP basically introduces a n
16 matches
Mail list logo
