gt; --
> View this message in context:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Firing-windows-multiple-times-tp8424p8994.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.
>
ded) for a 365-day window aggregating over a 30 day
window: if it filters out early fires, wouldn't it only produce new/unique
results every 30 days?
I very well may have misunderstood this solution.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com
y and feasibility of it.
>>
>> I noticed that FLIP-2 and FLIP-4 are still under discussion; is it
>> premature
>> to try to implement these enhancements? And would you be at all
>> willing/available to help me get up to speed?
>>
>> Thank you much!
>>
&
available to help me get up to speed?
>
> Thank you much!
>
>
>
> --
> View this message in context:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Firing-windows-multiple-times-tp8424p8975.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.
>
to try to implement these enhancements? And would you be at all
willing/available to help me get up to speed?
Thank you much!
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Firing-windows-multiple-times-tp8424p8975.html
Sent from the Apach
h can have an impact on the processing guarantees
> when a failure/recovery occurs if we don't do it carefully. Also, we're not
> particularly sophisticated yet with regard to avoiding unnecessary queries
> to the time series data.
>
> -Shannon
>
>
> From: Aljoscha K
-Shannon
From: Aljoscha Krettek mailto:aljos...@apache.org>>
Date: Friday, September 2, 2016 at 4:02 AM
To: "user@flink.apache.org<mailto:user@flink.apache.org>"
mailto:user@flink.apache.org>>
Subject: Re: Firing windows multiple times
I see, I didn't forget abo
ll
> be on average 0.5 months stale. A year-long window is even worse.
>
> -Shannon
>
> From: Aljoscha Krettek
> Date: Tuesday, August 30, 2016 at 9:08 AM
> To: Shannon Carey , "user@flink.apache.org" <
> user@flink.apache.org>
>
> Subject: Re: Firing
ia.com>>,
"user@flink.apache.org<mailto:user@flink.apache.org>"
mailto:user@flink.apache.org>>
Subject: Re: Firing windows multiple times
Hi,
I think this can be neatly expressed by using something like a tree of windowed
aggregations, i.e. you specify your smallest window c
rent watermark) on an ongoing basis.
> The windowing function would be responsible for evicting old data based on
> the current watermark.
>
> Does that make sense? Does it seem logical, or am I misunderstanding
> something about how Flink works?
>
> -Shannon
>
>
> Fr
ljoscha Krettek mailto:aljos...@apache.org>>
Date: Monday, August 29, 2016 at 3:56 AM
To: "user@flink.apache.org<mailto:user@flink.apache.org>"
mailto:user@flink.apache.org>>
Subject: Re: Firing windows multiple times
Hi,
that would certainly be possible? What do you th
ark to the window
> function metadata in FLIP-2?
>
> From: Shannon Carey
> Date: Friday, August 12, 2016 at 6:24 PM
> To: Aljoscha Krettek , "user@flink.apache.org" <
> user@flink.apache.org>
>
> Subject: Re: Firing windows multiple times
>
> Thanks Al
ser@flink.apache.org>"
mailto:user@flink.apache.org>>
Subject: Re: Firing windows multiple times
Thanks Aljoscha, I didn't know about those. Yes, they look like handy changes,
especially to enable flexible approaches for eviction. In particular, having
the current watermark availab
.org>"
mailto:user@flink.apache.org>>
Subject: Re: Firing windows multiple times
Hi,
there is already this FLIP:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-4+%3A+Enhance+Window+Evictor
which also links to a mailing list discussion. And this FLIP:
https://cwiki.apache.or
Hi,
there is already this FLIP:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-4+%3A+Enhance+Window+Evictor
which
also links to a mailing list discussion. And this FLIP:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-2+Extending+Window+Function+Metadata.
The former proposes to enhan
"If Window B is a Folding Window and does not have an evictor then it should
not keep the list of all received elements."
Agreed! Upon closer inspection, the behavior I'm describing is only present
when using EvictingWindowOperator, not when using WindowOperator. I misread
line 382 of WindowOpe
ermark, trigger's event timer is reached,
>fires and purges and emits current state as event z(time=1, count=2)
>9. Window B receives event, trigger waits for processing time delay,
>then executes fold() and emits event(time=1 => count=2), but internal
>Window stat
current state as event z(time=1, count=2)
>> Window B receives event, trigger waits for processing time delay, then
>> executes fold() and emits event(time=1 => count=2), but internal Window
>> state looks like [x(time=1, count=1), y(time=1, count=2), z(time=1, count=2)]
&
t; Does that explanation help interpret my original email?
>
> -Shannon
>
>
> From: Aljoscha Krettek mailto:aljos...@apache.org>>
> Date: Wednesday, August 10, 2016 at 12:18 PM
> To: "user@flink.apache.org <mailto:user@flink.apache.org>"
> mailto:user@
planation help interpret my original email?
-Shannon
From: Aljoscha Krettek mailto:aljos...@apache.org>>
Date: Wednesday, August 10, 2016 at 12:18 PM
To: "user@flink.apache.org<mailto:user@flink.apache.org>"
mailto:user@flink.apache.org>>
Subject: Re: Firing windows
Hi Aljoscha,
This looks like the bug that we discussed, as part of Enhance window
evictor JIRA
Thanks,
Vishnu
On Wed, Aug 10, 2016 at 1:18 PM, Aljoscha Krettek
wrote:
> Hi,
> from your mail I'm gathering that you are in fact using an Evictor, is
> that correct? If not, then the window operator
Hi,
from your mail I'm gathering that you are in fact using an Evictor, is that
correct? If not, then the window operator should not keep all the elements
ever received for a window but only the aggregated result.
Side note, there seems to be a bug in EvictingWindowOperator that causes
evicted ele
One unfortunate aspect of using a fold() instead of a window is that the fold
function has no knowledge of the watermarks. As a result, it is difficult to
ensure that only items before the current watermark are included in the
aggregation, and that old items are evicted correctly. This fact lend
I recently noticed something about windows: they retain (in state) every
element that they receive regardless of whether the user provides a fold/reduce
function. I can tell that such an approach is necessary in order for evictors
to work, but I'm not sure if there are other reasons.
I'll descr
24 matches
Mail list logo
