abian Hueske; Jörn Franke; user
Subject: Re: Flink application does not scale as expected, please help!
Hi,
as Fabian explained, if you exceed the number of slots on a single TM, then
Flink needs to deploy tasks on other TMs which causes a network transfer
between the sources and the mappers. This
*Cc:* Siew Wai Yow; Jörn Franke; user@flink.apache.org
> *Subject:* Re: Flink application does not scale as expected, please help!
>
> Hi all,
>
> Allow me to add some comments/questions on this issue that is very
> interesting.
> According to documentation [1] the pipeline exa
ing to know too. @Till do you mind to share
> your thoughts?
>
> Thank you guys!
>
> --
> *From:* Ovidiu-Cristian MARCU
> *Sent:* Monday, June 18, 2018 6:28 PM
> *To:* Fabian Hueske
> *Cc:* Siew Wai Yow; Jörn Franke; user@flink.apache.org
>
o you mind to share your
thoughts?
Thank you guys!
From: Ovidiu-Cristian MARCU
Sent: Monday, June 18, 2018 6:28 PM
To: Fabian Hueske
Cc: Siew Wai Yow; Jörn Franke; user@flink.apache.org
Subject: Re: Flink application does not scale as expected, please help!
Hi a
parallelism 16. Is
> this something expected? Thank you.
>
> Regards,
> Yow
>
> From: Fabian Hueske mailto:fhue...@gmail.com>>
> Sent: Monday, June 18, 2018 3:47 PM
> To: Siew Wai Yow
> Cc: Jörn Franke; user@flink.apache.org <mailto:user@flink.apache.org>
&g
day, June 18, 2018 3:47 PM
> *To:* Siew Wai Yow
> *Cc:* Jörn Franke; user@flink.apache.org
>
> *Subject:* Re: Flink application does not scale as expected, please help!
>
> Hi,
>
> Which Flink version are you using?
> Did you try to analyze the bottleneck of the application,
*additional info in bold.
From: Siew Wai Yow
Sent: Monday, June 18, 2018 3:57 PM
To: Fabian Hueske
Cc: Jörn Franke; user@flink.apache.org
Subject: Re: Flink application does not scale as expected, please help!
Hi Fabian,
We are using Flink 1.5.0. Any
anke; user@flink.apache.org
Subject: Re: Flink application does not scale as expected, please help!
Hi,
Which Flink version are you using?
Did you try to analyze the bottleneck of the application, i.e., is it CPU, disk
IO, or network bound?
Regarding the task scheduling. AFAIK, before version 1.5.0, Flin
rn Franke
> *Sent:* Saturday, June 16, 2018 6:03 PM
>
> *To:* Siew Wai Yow
> *Cc:* user@flink.apache.org
> *Subject:* Re: Flink application does not scale as expected, please help!
>
> Can you share the app source on gitlab, github or bitbucket etc?
>
> On 16. Jun 2018, at 11
Hi Jorn, Please find the source @https://github.com/swyow/flink_sample_git
Thank you!
From: Jörn Franke
Sent: Saturday, June 16, 2018 6:03 PM
To: Siew Wai Yow
Cc: user@flink.apache.org
Subject: Re: Flink application does not scale as expected, please help!
Can
gt; Yow
>
> From: Jörn Franke
> Sent: Saturday, June 16, 2018 4:46 PM
> To: Siew Wai Yow
> Cc: user@flink.apache.org
> Subject: Re: Flink application does not scale as expected, please help!
>
> How large is the input data? If the input data is very small then it does no
less I get?
From: Siew Wai Yow
Sent: Saturday, June 16, 2018 5:09 PM
To: Jörn Franke
Cc: user@flink.apache.org
Subject: Re: Flink application does not scale as expected, please help!
Hi Jorn, the input data is 1kb per record, in production it will have 10
billions
Siew Wai Yow
Cc: user@flink.apache.org
Subject: Re: Flink application does not scale as expected, please help!
How large is the input data? If the input data is very small then it does not
make sense to scale it even more. The larger the data is the more parallelism
you will have. You can modify
How large is the input data? If the input data is very small then it does not
make sense to scale it even more. The larger the data is the more parallelism
you will have. You can modify this behavior of course by changing the partition
on the Dataset.
> On 16. Jun 2018, at 10:41, Siew Wai Yow
14 matches
Mail list logo
