Yow Siew Wai created FLINK-9442:
-----------------------------------
Summary: Flink Scaling not working
Key: FLINK-9442
URL: https://issues.apache.org/jira/browse/FLINK-9442
Project: Flink
Issue Type: Bug
Reporter: Yow Siew Wai
Hi,
We are in the middle of testing scaling ability of Flink. But we found that
scaling not working, no matter increase more slot or increase number of Task
Manager. We would expect a linear, if not close-to-linear scaling performance
but the result even show degradation. Appreciated any comments.
Test Details,
-VMWare vsphere
-Just a simple pass through test,
- auto gen source 3mil records, each 1000k in size, parallelism=1
- source pass into next map operator, which just return the same record,
and sent counter to statsD, parallelism is in cases = 2,4,6
- 3 TM, total 6 slots(2/TM) each JM/TM has 32 vCPU, 100GB memory
- Result:
- 2 slots: 26 seconds, 3mil/26=115k TPS
- 4 slots: 23 seconds, 3mil/23=130k TPS
- 6 slots: 22 seconds, 3mil/22=136k TPS
As shown the scaling is almost nothing. Any clue? Thanks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
