Hi, Song, Guo,
We updated our cluster to 1.10.1 and the cluster.evenly-spread-out-slots
works pretty well now.
Thanks for your help!
Best regards,
Mu
On Wed, Jul 8, 2020 at 9:35 PM Mu Kong wrote:
> Hi Song, Guo,
>
> Thanks for the information.
> I will first upgrade our flink cluster to 1.10.0
Hi Song, Guo,
Thanks for the information.
I will first upgrade our flink cluster to 1.10.0 and try again.
Currently, we are encountering some dependency conflict issue, possibly
with tranquility. But that is another issue.
For your information, (also as I described in the previous email)
*What Fl
Hi Mu,
Regarding your questions.
- The feature `spread out tasks evenly across task managers` is
introduced in Flink 1.10.0, and backported to Flink 1.9.2, per the JIRA
ticket [1]. That means if you configure this option in Flink 1.9.0, it
should not take any effect.
- Please be awa
Hi, Mu,
AFAIK, this feature is added to 1.9.2. If you use 1.9.0, would you
like to upgrade your Flink distribution?
Best,
Yangze Guo
On Tue, Jul 7, 2020 at 8:33 PM Mu Kong wrote:
>
> Hi, Guo,
>
> Thanks for helping out.
>
> My application has a kafka source with 60 subtasks(parallelism), and we
Hi, Guo,
Thanks for helping out.
My application has a kafka source with 60 subtasks(parallelism), and we
have 15 task managers with 15 slots on each.
*Before I applied the cluster.evenly-spread-out-slots,* meaning it is set
to default false, the operator 'kafka source" has 11 subtasks allocated
Hi, Mu,
IIUC, cluster.evenly-spread-out-slots would fulfill your demand. Why
do you think it does the opposite of what you want. Do you run your
job in active mode? If so, cluster.evenly-spread-out-slots might not
work very well because there could be insufficient task managers when
request slot f
Hi community,
I'm running an application to consume data from kafka, and process it then
put data to the druid.
I wonder if there is a way where I can allocate the data source consuming
process evenly across the task manager to maximize the usage of the network
of task managers.
So, for example,
