Never mind Till figured out a way, instead of doing the aggregation in
reduce, I moved that logic to apply of the window function.
On Thu, Mar 24, 2016 at 11:33 PM, Balaji Rajagopalan <
balaji.rajagopa...@olacabs.com> wrote:
> Till,
>
> Thanks for your reply, may be I should have given more det
Till,
Thanks for your reply, may be I should have given more details. val
stream = DataStream(String,Int) is already windowed. Ideally I have all
the data that I need in my data stream, all my trying to do is like
HashMap[String,Int] from tuples(String,Int) , if reduce is not the best
solution,
Hi Balaji,
the output you see is the correct output since you're computing a
continuous reduce of the incoming data. Since you haven't defined a time
frame for your reduce computation you either would have to wait for all
eternity to output the final result or you output every time you've
generate
I have keyed input stream on DateStream(String,Int) and wrote a reduce on
the keyedStream. The reduce is simple one summing up the integer values of
the same key.
val stream = DataStream(String,Int)
val keyedStream = stream.keyBy(_._1).reduce( new MyReduceFunction)
keyedStream.print()
class MyRed
