I working with JSON data that has an array member. Im aggregating values into this using minute long windows.
I ran the app for ~10 minutes and watched it consume 40% of the memory on a box with 32G. It was still growing when I stopped it. At this point it had created ~800 values each of which was < 1Mb in size (owing to the limitations on message size set at the broker). (I wrote all the values into Redis so I could count them and check the aggregation). 1. Why is it consuming so much memory? 2. Is there a strategy for controlling this growth? I get that it's keeping every window open in case a new value shows up. Maybe some way to relax this using event time vs clock time?
