[
https://issues.apache.org/jira/browse/FLINK-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16513950#comment-16513950
]
Fabian Hueske commented on FLINK-9597:
--------------------------------------
Can add a few more details about the environment in which you are running the
job?
> Flink fail to scale!
> --------------------
>
> Key: FLINK-9597
> URL: https://issues.apache.org/jira/browse/FLINK-9597
> Project: Flink
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.5.0
> Reporter: swy
> Priority: Major
> Attachments: flink_app_parser_git.zip, scaleNotWork.png
>
>
> Hi, we found that our Flink application with simple logic, which using
> process function is not scale-able when scale from 8 parallelism onward even
> though with sufficient resources. Below it the result which is capped at
> ~250k TPS. No matter how we tune the parallelism of the operators it just not
> scale, same to increase source parallelism.
> Please refer to "scaleNotWork.png",
> 1. fixed source parallelism 4, other operators parallelism 8
> 2. fixed source parallelism 4, other operators parallelism 16
> 3. fixed source parallelism 4, other operators parallelism 32
> 4. fixed source parallelism 6, other operators parallelism 8
> 5. fixed source parallelism 6, other operators parallelism 16
> 6. fixed source parallelism 6, other operators parallelism 32
> 7. fixed source parallelism 6, other operators parallelism 64 performance
> worse than parallelism 32.
> Sample source code attached(flink_app_parser_git.zip). It is a simple
> program, parsing json record into object, and pass it to a empty logic
> Flink's process function. Rocksdb is in used, and the source is generated by
> the program itself. This could be reproduce easily.
> We choose Flink because of it scalability, but this is not the case now,
> appreciated if anyone could help as this is impacting our projects! thank you.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
