Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-11-02 Thread Manu Zhang
much > > > > > > >> like the old variant, only replacing the "Window" parameter by > > the > > > > > > context > > > > > > >> (which contains everything about the window). > > > > > > >> It could also be calle

Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-11-02 Thread Aljoscha Krettek
> the > > > > > context > > > > > >> (which contains everything about the window). > > > > > >> It could also be called "WindowInvocationContext" or so. > > > > > >> > > > > > >> The current variant looks t

Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-11-02 Thread Manu Zhang
ionContext" or so. > > > > >> > > > > >> The current variant looks too Hadoop to me ;-) Everything done on > > the > > > > >> context object, and messy mocking when creating tests. > > > > >> > >

Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-11-02 Thread Aljoscha Krettek
>> > > > >> On Mon, Jul 18, 2016 at 6:42 PM, Radu Tudoran < > > radu.tudo...@huawei.com> > > > >> wrote: > > > >> > > > >> > Hi, > > > >> > > > > >> > Sorry - I made a mistake - I wa

Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-08-01 Thread Aljoscha Krettek
messy mocking when creating tests. > > > >> > > > >> On Mon, Jul 18, 2016 at 6:42 PM, Radu Tudoran < > > radu.tudo...@huawei.com> > > > >> wrote: > > > >> > > > >> > Hi, > > > >> > > > >

Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-08-01 Thread Stephan Ewen
com> > > >> wrote: > > >> > > >> > Hi, > > >> > > > >> > Sorry - I made a mistake - I was thinking of getting access to the > > >> > collection (mist-read :) collector) of events in the window buffer > in > > >> > order

Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-07-31 Thread Aljoscha Krettek
t; >> > Sorry - I made a mistake - I was thinking of getting access to the > >> > collection (mist-read :) collector) of events in the window buffer in > >> > order to be able to delete/evict some of them which are not necessary > the > >> > last ones. > >&g

Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-07-20 Thread Maximilian Michels
t have a logical order. One example is if you have >> > > high rates of the events. Nevertheless, even if now is not the time >> > > for this, I think in the future we can imagine having also some data >> > > structures that offer some ordering. It can save some com

Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-07-19 Thread Aljoscha Krettek
imagine having also some data > > > structures that offer some ordering. It can save some computation > > > efforts later in the functions for some use cases. > > > > > > > > > Best regards, > > > > > > > > > -Original Mess

Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-07-19 Thread Stephan Ewen
91588344173 > > > > > > HUAWEI TECHNOLOGIES Duesseldorf GmbH Hansaallee 205, 40549 > > > Düsseldorf, Germany, www.huawei.com Registered > > > Office: Düsseldorf, Register Court Düsseldorf, HRB 56063, Managing > > > Director: Bo PENG, Wanzho

RE: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-07-18 Thread Radu Tudoran
[mailto:aljos...@apache.org] Sent: Monday, July 18, 2016 5:54 PM To: dev@flink.apache.org Subject: Re: [DISCUSS] FLIP-2 Extending Window Function Metadata What about the collector? This is only used for emitting elements to the downstream operation. On Mon, 18 Jul 2016 at 17:52 Radu Tudoran wrote: >

Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-07-18 Thread Aljoscha Krettek
an save some computation efforts later in the functions > for some use cases. > > > Best regards, > > > -Original Message- > From: Aljoscha Krettek [mailto:aljos...@apache.org] > Sent: Monday, July 18, 2016 3:45 PM > To: dev@flink.apache.org > Subject: Re: [DISCUSS]

RE: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-07-18 Thread Radu Tudoran
@flink.apache.org Subject: Re: [DISCUSS] FLIP-2 Extending Window Function Metadata I incorporated the changes. The proposed interface of ProcessWindowFunction is now this: public abstract class ProcessWindowFunction implements Function { public abstract void process(KEY key, Iterable

Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-07-18 Thread Aljoscha Krettek
partial disclosure, reproduction, > or dissemination) by persons other than the intended recipient(s) is > prohibited. If you receive this e-mail in error, please notify the sender > by phone or email immediately and delete it! > > > -Original Message- > From: Aljoscha Krett

RE: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-07-13 Thread Radu Tudoran
al Message- From: Aljoscha Krettek [mailto:aljos...@apache.org] Sent: Wednesday, July 13, 2016 2:24 PM To: dev@flink.apache.org Subject: Re: [DISCUSS] FLIP-2 Extending Window Function Metadata Sure, I also thought about this but went for the "extreme" initially. If no-one objects I'

Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-07-13 Thread Aljoscha Krettek
Sure, I also thought about this but went for the "extreme" initially. If no-one objects I'll update the doc in a bit. On Wed, 13 Jul 2016 at 14:17 Stephan Ewen wrote: > Thanks for opening this. > > I see the need for having an extensible context object for window function > invocations, but i th

Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-07-13 Thread Stephan Ewen
Thanks for opening this. I see the need for having an extensible context object for window function invocations, but i think hiding every parameter in the context is a bit unnatural. How about having a function "apply(Key, Values, WindowContext, Collector)" ? It should be possible to write the st

[DISCUSS] FLIP-2 Extending Window Function Metadata

2016-07-13 Thread Aljoscha Krettek
Hi, this is a proposal to introduce a new interface for the window function to make it more extensible for the future where we might want to provide additional information about why a window fired to the user function: https://cwiki.apache.org/confluence/display/FLINK/FLIP-2+Extending+Window+Funct