Hi! I also think it's a GC problem. In the KeySelector I don't instantiate any object. It's a simple toString method call. In the mapWindow I create new objects. But I'm doing the same in other map operators, too. They don't slow down the execution. Only with this construct the execution is slowed down.
I watched on the memory footprint of my program. Once with the code construct I wrote and once without. The memory characteristic were the same. The CPU usage also ... I don't have an explanation. But I don't think it comes from my operator functions ... Cheers Rico. > Am 07.09.2015 um 22:43 schrieb Martin Neumann <mneum...@sics.se>: > > Hej, > > This sounds like it could be a garbage collection problem. Do you instantiate > any classes inside any of the operators (e.g. in the KeySelector). You can > also try to run it locally and use something like jstat to rule this out. > > cheers Martin > >> On Mon, Sep 7, 2015 at 12:00 PM, Rico Bergmann <i...@ricobergmann.de> wrote: >> Hi! >> >> While working with grouping and windowing I encountered a strange behavior. >> I'm doing: >>> dataStream.groupBy(KeySelector).window(Time.of(x, >>> TimeUnit.SECONDS)).mapWindow(toString).flatten() >> >> When I run the program containing this snippet it initially outputs data at >> a rate around 150 events per sec. (That is roughly the input rate for the >> program). After about 10-30 minutes the rate drops down below 5 events per >> sec. This leads to event delivery offsets getting bigger and bigger ... >> >> Any explanation for this? I know you are reworking the streaming API. But it >> would be useful to know, why this happens ... >> >> Cheers. Rico. >
