Re: About delta awareness caches

2017-01-24 Thread Aljoscha Krettek
Hi, in fact, this was just merged: https://issues.apache.org/jira/browse/FLINK-5582. It will be released as part of Flink 1.3 in roughly 4 months. The feature is still a bit rough around the edges and needs some follow-up work, however. Cheers, Aljoscha On Thu, 12 Jan 2017 at 11:10 Xingcan wrote

Re: About delta awareness caches

2017-01-12 Thread Xingcan
Hi, Aljoscha Thanks for your explanation. About the Storm windows simulation, we had tried your suggestion and gave up due to its complexity and sort of "reinventing the wheel". Without considering the performance, most of our business-logic code have already been transformed to the "Flink style"

Re: About delta awareness caches

2017-01-11 Thread Aljoscha Krettek
Hi, (I'm just getting back from holidays, therefore the slow response. Sorry for that.) I think you can simulate the way Storm windows work by using a GlobalWindows assigner and having a custom Trigger and/or Evictor and also some special logic in your WindowFunction. About mergeable state, we're

Re: About delta awareness caches

2016-12-21 Thread xingcan
Hi Aljoscha, First of all, sorry for that I missed your prompt reply : ( In these days, I've been learning the implementation mechanism of window in Flink. I think the main difference between the window in Storm and Flink (from the API level) is that, Storm maintains only one window while Flink

Re: About delta awareness caches

2016-12-01 Thread Aljoscha Krettek
I'm not aware of how windows work in Storm. If you could maybe give some details on your use case we could figure out together how that would map to Flink windows. Cheers, Aljoscha On Tue, 29 Nov 2016 at 15:47 xingcan wrote: > Hi all, > > Recently I tried to transfer some old applications from

About delta awareness caches

2016-11-29 Thread xingcan
Hi all, Recently I tried to transfer some old applications from Storm to Flink. In Storm, the window implementation (TupleWindow) gets two methods named getNew() and getExpired() which supply the delta information of a window and therefore we wrote some stateful caches that are aware of them. Howe