custom partitioner which can
control the logic of keyBy distribution based on pre-defined cache distribution
in nodes?
Best,
Zhijiang
--
From:Navneeth Krishnan
Send Time:2020年9月23日(星期三) 02:21
To:user
Subject:Adaptive load balancing
memory
problems for us and we would like to explore some options to mitigate the
current limitations.
Is there a way to group a set of keys and send to a set of nodes so that we
don't have to replicate the cache data on all nodes?
Has someone tried implementing hashing with adaptive load balanci
Theo
-Original Message-
From: KristoffSC
Sent: Dienstag, 17. Dezember 2019 23:35
To: user@flink.apache.org
Subject: Keyed stream, parallelism, load balancing and ensuring that the
same key go to the same Task Manager and task slot
Hi community,
I'm trying to build a PoC pipeline for my project
Hi :)
Any thoughts about this?
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Hi community,
I'm trying to build a PoC pipeline for my project and I have few questions
regarding load balancing between task managers and ensuring that keyed
stream events for the same key will go to the same Task Manager (hence the
same task slot).
Lets assume that we have 3 task manage
.n4.nabble.com/load-balancing-of-keys-to-operators-tp12303p12308.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
Thanx for your response.
When using time windows, doesn`t flink know the load per window? I have
observed this behavior in windows as well.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/load-balancing-of-keys-to-operators-tp12303p12308
context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/load-balancing-of-keys-to-operators-tp12303.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
-user-mailing-list-archive.2336050.n4.nabble.com/load-balancing-of-keys-to-operators-tp12303.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
, I noticed the PKG strategy and
maybe it will come in handy in some other place :)
So, thanks again for the pointers!
Cheers,
Sebastian
From: Gianmarco De Francisci Morales [mailto:g...@apache.org]
Sent: Freitag, 12. Juni 2015 19:02
To: user@flink.apache.org
Subject: Re: Load balancing
Hi
e, I am missing something, but doesn’t
> this assumption render PKG inapplicable to my case? Objections to that are
> of course welcome :)
>
>
>
> Cheers,
>
> Sebastian
>
>
>
> *From:* Gianmarco De Francisci Morales [mailto:g...@apache.org]
> *Sent:* Mittwoch, 10
:40
To: user@flink.apache.org
Subject: Re: Load balancing
We have been working on an adaptive load balancing strategy that would address
exactly the issue you point out.
FLINK-1725 is the starting point for the integration.
Cheers,
--
Gianmarco
On 9 June 2015 at 20:31, Fabian Hueske
mailto:fhue
We have been working on an adaptive load balancing strategy that would
address exactly the issue you point out.
FLINK-1725 is the starting point for the integration.
Cheers,
--
Gianmarco
On 9 June 2015 at 20:31, Fabian Hueske wrote:
> Hi Sebastian,
>
> I agree, shuffling only
Hi Sebastian,
I agree, shuffling only specific elements would be a very useful feature,
but unfortunately it's not supported (yet).
Would you like to open a JIRA for that?
Cheers, Fabian
2015-06-09 17:22 GMT+02:00 Kruse, Sebastian :
> Hi folks,
>
>
>
> I would like to do
Hi folks,
I would like to do some load balancing within one of my Flink jobs to achieve
good scalability. The rebalance() method is not applicable in my case, as the
runtime is dominated by the processing of very few larger elements in my
dataset. Hence, I need to distribute the processing
15 matches
Mail list logo
