[GitHub] [flink] wsry commented on issue #11351: [FLINK-16404][runtime] Avoid caching buffers for blocked input channels before barrier alignment

[GitBox]

wsry commented on issue #11351:
URL: https://github.com/apache/flink/pull/11351#issuecomment-617578042


   @zhijiangW I did some performance tests, all tests are configured 1s 
checkpoint Interval, the test job has three vertices (source -> map -> sink) 
and up/down streams are connected by RemoteInputChannel (slot sharing is 
disabled by setting different slot sharing group, rebalance partitioner is used 
to break operator chain),  the followings are the results (can be reproduced) 
and from the results we can say there is no performance issue:
   
   results of on single node and 1 parallelism
   without this patch:
   ```
   Benchmark                                                                
Mode  Cnt     Score     Error   Units
   RemoteChannelThroughputBenchmark.remoteChannelWithCheckpointThroughput  
thrpt   30  4811.630 ± 546.672  ops/ms
   ```
   with this patch:
   ```
   Benchmark                                                                
Mode  Cnt     Score     Error   Units
   RemoteChannelThroughputBenchmark.remoteChannelWithCheckpointThroughput  
thrpt   30  4904.883 ± 568.675  ops/ms
   ```
   
   results of on single node and 10 parallelism
   without this patch:
   ```
   Benchmark                                                                
Mode  Cnt      Score      Error   Units
   RemoteChannelThroughputBenchmark.remoteChannelWithCheckpointThroughput  
thrpt   30  18059.951 ± 1804.108  ops/ms
   ```
   with this patch:
   ```
   Benchmark                                                                
Mode  Cnt      Score      Error   Units
   RemoteChannelThroughputBenchmark.remoteChannelWithCheckpointThroughput  
thrpt   30  18377.411 ± 1774.382  ops/ms
   ```
   
   results of on single node and 100 parallelism
   without this patch:
   ```
   Benchmark                                                                
Mode  Cnt      Score      Error   Units
   RemoteChannelThroughputBenchmark.remoteChannelWithCheckpointThroughput  
thrpt   30  52883.613 ± 2047.977  ops/ms
   ```
   with this patch:
   ```
   Benchmark                                                                
Mode  Cnt      Score      Error   Units
   RemoteChannelThroughputBenchmark.remoteChannelWithCheckpointThroughput  
thrpt   30  51351.670 ± 2412.904  ops/ms
   ```
   
   results of on cluster and 100 parallelism（fixed number of total records）
   without this patch:
   
![image](https://user-images.githubusercontent.com/5469568/79942662-e9776480-8499-11ea-9f8b-701fdf6a7c38.png)
   
   with this patch:
   
![image](https://user-images.githubusercontent.com/5469568/79942686-fc8a3480-8499-11ea-9e7c-258408c8ec2a.png)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org