Re: load balancing of keys to operators

2017-03-21 Thread Timo Walther
I think it very depends on your use case, maybe you can use combiner first to reduce the amount of records per key. Maybe you can explain your application a bit more (which window, type of aggregations). It often helps e.g. to introduce an artifical key und merge the result of multiple windows

Re: load balancing of keys to operators

2017-03-20 Thread Sonex
Thanx for your response. When using time windows, doesn`t flink know the load per window? I have observed this behavior in windows as well. -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/load-balancing-of-keys-to-operators-tp12303p12308.htm

Re: load balancing of keys to operators

2017-03-20 Thread Timo Walther
Hi, using keyBy Flink ensures that every set of records with same key is send to the same operator, otherwise it would not be possible to process them as a whole. It depends on your use case if it is also ok that another operator processes parts of this set of records. You can implement you o