Fantastic. Sounds like things are moving in the right direction. I'm hoping
this will be tiered storage.
Thanks!
On Fri, 15 Jan 2016, 17:04 Aljoscha Krettek wrote:
> Hi,
> don’t worry, it’s good to get questions about this stuff. :D
>
> You are right, if Flink is not clever about the state your
Hi,
don’t worry, it’s good to get questions about this stuff. :D
You are right, if Flink is not clever about the state your JVMs can run out of
memory and blow up. We are currently working on several things that should make
this more robust:
1) Put Flink Windows on Flink’s partitioned state abst
Hi Aljoscha,
Just thinking on the EventTimeTrigger example you provided, and I'm going
to apologise in advance for taking more of your time!, but I'm thinking
that should I go down that route any long allowedLateness, we'll run into
issues with memory use, unless Flink is smart enough, configurab
Hi Aljoscha,
Thanks for the info!
Andy
On Fri, 15 Jan 2016 at 10:12 Aljoscha Krettek wrote:
> Hi,
> I imagine you are taking about CountTrigger, DeltaTrigger, and
> Continuous*Trigger. For these we never purge. They are a leftover artifact
> from an earlier approach to implementing windowing s
Hi,
I imagine you are taking about CountTrigger, DeltaTrigger, and
Continuous*Trigger. For these we never purge. They are a leftover artifact from
an earlier approach to implementing windowing strategies that was inspired by
IBM InfoSphere streams. Here, all triggers are essentially accumulating
Thanks Aljoscha, that's very enlightening.
Can you please also explain what the default behaviour is? I.e. if I use
one if the accumulating inbuilt triggers, when does the state get purged?
(With your info I can now probably work things out, but you may give more
insight :)
Also, are there plans
Hi,
the window contents are stored in state managed by the window operator at all
times until they are purged by a Trigger returning PURGE from one of its on*()
methods.
Out of the box, Flink does not have something akin to the lateness and cleanup
of Google Dataflow. You can, however implement
Hi,
I'm trying to understand how the lifecycle of messages / state is managed
by Flink, but I'm failing to find any documentation.
Specially, if I'm using a windowed stream and a type of trigger that retain
the elements of the window to allow for processing of late data e.g.
ContinousEventTimeTri
