I think it very depends on your use case, maybe you can use combiner
first to reduce the amount of records per key. Maybe you can explain
your application a bit more (which window, type of aggregations).
It often helps e.g. to introduce an artifical key und merge the result
of multiple windows
Thanx for your response.
When using time windows, doesn`t flink know the load per window? I have
observed this behavior in windows as well.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/load-balancing-of-keys-to-operators-tp12303p12308.htm
Hi,
using keyBy Flink ensures that every set of records with same key is
send to the same operator, otherwise it would not be possible to process
them as a whole. It depends on your use case if it is also ok that
another operator processes parts of this set of records. You can
implement you o
, I noticed the PKG strategy and
maybe it will come in handy in some other place :)
So, thanks again for the pointers!
Cheers,
Sebastian
From: Gianmarco De Francisci Morales [mailto:g...@apache.org]
Sent: Freitag, 12. Juni 2015 19:02
To: user@flink.apache.org
Subject: Re: Load balancing
Hi
e, I am missing something, but doesn’t
> this assumption render PKG inapplicable to my case? Objections to that are
> of course welcome :)
>
>
>
> Cheers,
>
> Sebastian
>
>
>
> *From:* Gianmarco De Francisci Morales [mailto:g...@apache.org]
> *Sent:* Mittwoch, 10
:40
To: user@flink.apache.org
Subject: Re: Load balancing
We have been working on an adaptive load balancing strategy that would address
exactly the issue you point out.
FLINK-1725 is the starting point for the integration.
Cheers,
--
Gianmarco
On 9 June 2015 at 20:31, Fabian Hueske
mailto:fhue
We have been working on an adaptive load balancing strategy that would
address exactly the issue you point out.
FLINK-1725 is the starting point for the integration.
Cheers,
--
Gianmarco
On 9 June 2015 at 20:31, Fabian Hueske wrote:
> Hi Sebastian,
>
> I agree, shuffling only specific elements
Hi Sebastian,
I agree, shuffling only specific elements would be a very useful feature,
but unfortunately it's not supported (yet).
Would you like to open a JIRA for that?
Cheers, Fabian
2015-06-09 17:22 GMT+02:00 Kruse, Sebastian :
> Hi folks,
>
>
>
> I would like to do some load balancing wi
