Ah ok. And by distributed you mean that the element that starts the window can be processed on a different machine than the element that finishes the window?
On Wed, Jun 3, 2015 at 12:11 PM, Gyula Fóra <gyula.f...@gmail.com> wrote: > This is not connected to the current implementation. So lets not talk about > that. > > This is about theoretical limits to support distributed delta policies > which has far reaching implications for the windowing policies one can > implement in a prallel way. > > But you are welcome to throw in any constructive ideas :) > On Wed, Jun 3, 2015 at 11:49 AM Aljoscha Krettek <aljos...@apache.org> > wrote: > >> Part of the reason for my question is this: >> https://issues.apache.org/jira/browse/FLINK-1967. Especially my latest >> comment there. If we want this, I think we have to overhaul the >> windowing system anyways and then it doesn't make sense to explore >> complicated workarounds for the current system. >> >> On Wed, Jun 3, 2015 at 11:07 AM, Gyula Fóra <gyula.f...@gmail.com> wrote: >> > There are simple ways of implementing it in a non-distributed or >> > inconsistent fashion. >> > On Wed, Jun 3, 2015 at 8:55 AM Aljoscha Krettek <aljos...@apache.org> >> wrote: >> > >> >> This already sounds awfully complicated. Is there no other way to >> >> implement the delta windows? >> >> >> >> On Wed, Jun 3, 2015 at 7:52 AM, Gyula Fóra <gyula.f...@gmail.com> >> wrote: >> >> > Hi Ufuk, >> >> > >> >> > In the concrete use case I have in mind I only want to send events to >> >> > another subtask of the same task vertex. >> >> > >> >> > Specifically: if we want to do distributed delta based windows we >> need to >> >> > send after every trigger the element that has triggered the current >> >> window. >> >> > So practically I want to broadcast some event regularly to all >> subtasks >> >> of >> >> > the same operator. >> >> > >> >> > In this case the operators would wait until they receive this event >> so we >> >> > need to make sure that this event sending is not blocked by the actual >> >> > records. >> >> > >> >> > Gyula >> >> > >> >> > On Tuesday, June 2, 2015, Ufuk Celebi <u...@apache.org> wrote: >> >> > >> >> >> >> >> >> On 02 Jun 2015, at 22:45, Gyula Fóra <gyf...@apache.org >> <javascript:;>> >> >> >> wrote: >> >> >> > I am wondering, what is the suggested way to send some events >> >> directly to >> >> >> > another parallel instance in a flink job? For example from one >> mapper >> >> to >> >> >> > another mapper (of the same operator). >> >> >> > >> >> >> > Do we have any internal support for this? The first thing that we >> >> thought >> >> >> > of is iterations but that is clearly an overkill. >> >> >> >> >> >> There is no support for this at the moment. Any parallel instance? >> Or a >> >> >> subtask instance of the same task? >> >> >> >> >> >> Can you provide more input on the use case? It is certainly possible >> to >> >> >> add support for this. >> >> >> >> >> >> If the events don't need to be inline with the records, we can easily >> >> >> setup the TaskEventDispatcher as a separate actor (or extend the task >> >> >> manager) to process both backwards flowing events and in general any >> >> events >> >> >> that don't need to be inline with the records. The task deployment >> >> >> descriptors need to be extended with the extra parallel instance >> >> >> information. >> >> >> >> >> >> – Ufuk >> >> >>
