> Thanks and Regards,
> > >> >> >> > > > >>> > Vishnu Viswanath,
> > >> >> >> > > > >>> >
> > >> >> >> > > > >>> > On Fri, Jul 8, 2016 at 6:04 AM, Aljoscha Kre
> >> >> >> > > > >>> > > come to think of it, the right place to put such
> >> checks
> >> >> is
> >> >> >> > > actually
> >> >> >> > > > >>> the
> >> >> >> > > > >>> > &g
> have
>> >> >> > > to
>> >> >> > > > >>> do the
>> >> >> > > > >>> > > processing and you have good separation of concerns.
>> Does
>> >> >> that
>> >> &
; > Aljoscha
> >> >> > > > >>> > >
> >> >> > > > >>> > > On Thu, 7 Jul 2016 at 23:56 Vishnu Viswanath <
> >> >> > > > >>> > vishnu.viswanat...@gmail.com
> >> >> &
by the window function: I think it will be nice if we
>> can
>> >> > come
>> >> > > > up
>> >> > > > >>> with
>> >> > > > >>> > > > something that is LESS coupled, because I can think of
>> >> &g
...@huawei.com
> >> > > > >>> > > > > Mobile: +49 15209084330
> >> > > > >>> > > > > Telephone: +49 891588344173
> >> > > > >>> > > > >
> >&g
gt; > > Thanks and Regards,
>> > > > >>> > > > Vishnu Viswanath,
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > > > On Thu, Jul 7, 2016 at 9:19 AM, Rad
gt; > > >>> > trigger,
> > > > >>> > > > > window, apply function). However, I think it's possible
> > just
> > > to
> > > > >>> > create
> > > > >>> > > > more
> > >
abled or more functionality (e.g., access to the each
> > element
> > > >>> in
> > > >>> > the
> > > >>> > > > > evictor ; possibility to delete or mark for eviction
> elements
> > > in
> > > >>> the
complexity to sort (the worst case) is very unlikely. In fact
> > you
> > >>> > have
> > >>> > > > > almost sorted items/arrays. Moreover, if you consider that in
> > >>> > > iteration X
> > >>> > > &
>>> > might
> >>> > > > be
> >>> > > > > significant smaller then N (elements that exists). Then using
> an
> >>> > > > insertion
> >>> > > > > sort for these new elements you would have
> > > If M is proportional to N then you can sort M and use merge sort
>>> for
>>> > > > > combining.
>>> > > > >
>>> > > > >
>>> > > > > Dr. Radu Tudoran
>>> > > > > Research Engin
> > n
>> > > > > (plus the work of actually deciding what to evict) of the sort
>> based
>> > > > > strategy.
>> > > > >
>> > > > > I might be wrong, though, and there could be a valid use-case not
>> &g
6063,
> > > > > Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN
> > > > > This e-mail and its attachments contain confidential information
> from
> > > > > HUAWEI, which is intended only for the person or entity whose
> address
> >
ded recipient(s) is
> > > > prohibited. If you receive this e-mail in error, please notify the
> > sender
> > > > by phone or email immediately and delete it!
> > > >
> > > >
> > > > -Original Message-
> > > &g
ly 07, 2016 11:59 AM
> > > To: dev@flink.apache.org
> > > Subject: 答复: [DISCUSS] Enhance Window Evictor in Flink
> > >
> > > HI,
> > > I think it is necessary to support sorted window, which can avoid
> > scanning
> > > all the elements of window while trying
t; scanning
> > all the elements of window while trying to evicting element, which may
> cost
> > many IO operations, such as querying DBs to get elements from state.
> > What's more, when an window aggregation function is invertible, such as
> > sum, which can be u
ormance of window aggregation, if evictor can
> trigger update of window aggregation state by some mechanism.
>
> Best Wishes!
> ---
> wenlong.lwl
>
> -邮件原件-
> 发件人: Aljoscha Krettek [mailto:aljos...@apache.org]
> 发送时间: 2016年7月7日 17:32
> 收件人: dev@flink.apache.org
ek [mailto:aljos...@apache.org]
发送时间: 2016年7月7日 17:32
收件人: dev@flink.apache.org
主题: Re: [DISCUSS] Enhance Window Evictor in Flink
Hi,
regarding "sorting the window by event time": I also considered this but in the
end I don't think it's necessary. Sorting is rather expensive an
ut not limited to, total or partial disclosure, reproduction,
> or dissemination) by persons other than the intended recipient(s) is
> prohibited. If you receive this e-mail in error, please notify the sender
> by phone or email immediately and delete it!
>
> -Original Message-----
&
nath [mailto:vishnu.viswanat...@gmail.com]
Sent: Thursday, July 07, 2016 1:28 AM
To: Dev
Subject: Re: [DISCUSS] Enhance Window Evictor in Flink
Thank you Maxim and Aljoscha.
Yes the beforeEvict and afterEvict should able address point 3.
I have one more use case in my mind (which I might have to do in the l
Thank you Maxim and Aljoscha.
Yes the beforeEvict and afterEvict should able address point 3.
I have one more use case in my mind (which I might have to do in the later
stages of POC).
What if the `evictAfter` should behave differently based on the window
function.
For example.
I have a window t
Actually for such evictor to be useful the window should be sorted by some
field, usually event time. What do you think about adding sorted window
abstraction?
On Wed, Jul 6, 2016 at 11:36 AM, Aljoscha Krettek
wrote:
> @Maxim: That's perfect I didn't think about using Iterator.remove() for
> tha
@Maxim: That's perfect I didn't think about using Iterator.remove() for
that. I'll update the doc. What do you think Vishnu? This should also cover
your before/after case nicely.
@Vishnu: The steps would be these:
- Converge on a design in this discussion
- Add a Jira issue here: https://issues.
The new API forces iteration through every element of the buffer even if a
single value to be evicted. What about implementing Iterator.remove()
method for elements? The API would look like:
public interface Evictor extends Serializable {
/**
* Optionally evicts elements. Called before wi
Hi Aljoscha,
Thanks. Yes the new interface seems to address points 1 and 2. of
*1) I am having a use case where I have to create a custom Evictor that
will evict elements from the window based on the value (e.g., if I have
elements are of case class Item(id: Int, type:String) then evict elements
26 matches
Mail list logo
