Re: Writing a reliable Flink source for a NON-replay-able queue/protocol that supports message ACKs

2017-05-07 Thread Martin Eden
Hi Kostas, Thanks for pointing me in the right direction. I have gone and extended MessageAcknowledgingSourceBase. It was quite easy to do. I have however some follow-up questions about the guarantees it gives and testing my solution. 1. Guarantees: *Questions:* a. When the acknowledgeIDs meth

AllWindowed vs Windowed with 1 key

2017-05-07 Thread Adrienne Kole
Hi, I am doing simple aggregation with a keyed and global windows in flink. When I compare the keyed window aggregation with 1 key and global window (which has parallelism 1) I would expect that both of them would have similar performance. However, keyed stream with 1 key performs with 2x more th

Can ValueState use generics?

2017-05-07 Thread yunfan123
My process function is like : private static class MergeFunction extends RichProcessFunction, Tuple2> { private ValueState> state; @Override @SuppressWarnings("unchecked") public void open(Configuration parameters) throws Exception { state = getRun

Re: ConnectedStream keyby issues

2017-05-07 Thread yunfan123
But what happened if some data can't be merged forever ? The state will be saved forever? Can I set a timeout? -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ConnectedStream-keyby-issues-tp12999p13037.html Sent from the Apache Flink User Ma