Re: Aggregator State in Keyed Windowed Stream

2018-09-10 Thread Ning Shi
Hi Andrey, > the answer is yes, it is backed by state backend (should be RocksDB if you > configure it), > you can trace it through these method calls: > > sourceStream.keyBy(…) > .timeWindow(Time.seconds(…)) > .trigger(CountTrigger.of(…)) > gives you WindowedStream, > WindowedStream.aggregate(ne

Re: Aggregator State in Keyed Windowed Stream

2018-09-10 Thread Ning Shi
> Back to my first question, is the accumulator state backed by RocksDB state > backend? If so, I don’t need to use rich function for the aggregate function. I did some testing and code reading. To answer my own question, the accumulator state seems to be managed by RocksDB if I use it as the sta

Re: Aggregator State in Keyed Windowed Stream

2018-09-10 Thread Andrey Zagrebin
Hi Ning, > Back to my first question, is the accumulator state backed by RocksDB state > backend? If so, I don’t need to use rich function for the aggregate function. the answer is yes, it is backed by state backend (should be RocksDB if you configure it), you can trace it through these method

Re: Aggregator State in Keyed Windowed Stream

2018-09-10 Thread Ning Shi
Hi Vino, > If you need access to the state API, you can consider using > ProcessWindowFunction[1], which allows you to use ProcessWindowFunction. I was hoping that I could use the aggregate function to do incremental aggregation. My understanding is that ProcessWindowFunction either has to loop

Re: Aggregator State in Keyed Windowed Stream

2018-09-10 Thread vino yang
In addition : ProcessWindowFunction extends AbstractRichFunction, through getRuntimeContext,you can access keyed state API. vino yang 于2018年9月10日周一 下午3:19写道: > Hi Ning, > > Answer you question: > > *And why is rich functions not allowed here?* > > If you need access to the state API, you can co

Re: Aggregator State in Keyed Windowed Stream

2018-09-10 Thread vino yang
Hi Ning, Answer you question: *And why is rich functions not allowed here?* If you need access to the state API, you can consider using ProcessWindowFunction[1], which allows you to use ProcessWindowFunction. Thanks, vino. [1]: https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/st