Hi,
If it is to extend the Context to pass more information between the stages of
processing a window (triggering -> process -> eviction), why not adding also a
"EvictionInfo"? I think this might actually help with the issues discussed in
the tread related to the eviction policy. I could imagine using this parameter
to pass the conditions, from the processing stage to the evictor, about what
events to be eliminated.
public abstract class Context {
public abstract EvictionInfo evictionInfo();
...
public abstract KEY key();
public abstract W window();
public abstract int id();
public abstract FiringInfo firingInfo();
public abstract Iterable<IN> elements();
public abstract void output(OUT value);
}
Also on a slightly unrelated issue - how hard it would be to introduce
different types of buffers for the windows. Currently the existing one is
behaving (when under processing) similar with a FIFO queue (in the sense that
you need to start from beginning, from the oldest element). How about enabling
for example also LIFO behavior (start iterating through the list from the most
recent element). As in the source queues or stacks are not actually used,
perhaps we can just pass policies to the iterator - or have custom itrators
Dr. Radu Tudoran
Research Engineer - Big Data Expert
IT R&D Division
HUAWEI TECHNOLOGIES Duesseldorf GmbH
European Research Center
Riesstrasse 25, 80992 München
E-mail: radu.tudo...@huawei.com
Mobile: +49 15209084330
Telephone: +49 891588344173
HUAWEI TECHNOLOGIES Duesseldorf GmbH
Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com
Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
Managing Director: Bo PENG, Wanzhou MENG, Lifang CHEN
Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN
This e-mail and its attachments contain confidential information from HUAWEI,
which is intended only for the person or entity whose address is listed above.
Any use of the information contained herein in any way (including, but not
limited to, total or partial disclosure, reproduction, or dissemination) by
persons other than the intended recipient(s) is prohibited. If you receive this
e-mail in error, please notify the sender by phone or email immediately and
delete it!
-----Original Message-----
From: Aljoscha Krettek [mailto:aljos...@apache.org]
Sent: Wednesday, July 13, 2016 2:24 PM
To: dev@flink.apache.org
Subject: Re: [DISCUSS] FLIP-2 Extending Window Function Metadata
Sure, I also thought about this but went for the "extreme" initially. If no-one
objects I'll update the doc in a bit.
On Wed, 13 Jul 2016 at 14:17 Stephan Ewen <se...@apache.org> wrote:
> Thanks for opening this.
>
> I see the need for having an extensible context object for window
> function invocations, but i think hiding every parameter in the
> context is a bit unnatural.
>
> How about having a function "apply(Key, Values, WindowContext, Collector)"
> ?
> It should be possible to write the straightforward use cases without
> accessing the context object.
>
>
>
> On Wed, Jul 13, 2016 at 1:56 PM, Aljoscha Krettek
> <aljos...@apache.org>
> wrote:
>
> > Hi,
> > this is a proposal to introduce a new interface for the window
> > function
> to
> > make it more extensible for the future where we might want to
> > provide additional information about why a window fired to the user
> > function:
> >
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-2+Extending+Win
> dow+Function+Metadata
> >
> > I'd appreciate your thoughts!
> >
> > Cheers,
> > Aljoscha
> >
>
