Re: How to merge messages from all partitions

2016-12-12 Thread Till Rohrmann
Hi Naveen, you would have to apply an all reduce after you’ve aggregated the values by key. metrics... timeWindowAll(Time.seconds(30)).fold(..., ...); However, be aware that this all window fold operation will be executed by a single operator (parallelism 1). You could also do this all window o

How to merge messages from all partitions

2016-12-09 Thread Naveen Tirupattur
HI, I am trying to group messages by message name, timestamp and then perform aggregation on message value. My window function looks like below metrics.keyBy("metricName") .keyBy("timeStamp") .timeWindow(Time.seconds(30)) .trigger(ProcessingTimeTrigger.create()) .fold(new Tuple3(