tartarus created FLINK-31655:
--------------------------------
Summary: Adaptive Channel selection for partitioner
Key: FLINK-31655
URL: https://issues.apache.org/jira/browse/FLINK-31655
Project: Flink
Issue Type: Improvement
Components: Runtime / Task
Reporter: tartarus
In Flink, if the upstream and downstream operator parallelism is not the same,
then by default the RebalancePartitioner will be used to select the target
channel.
In our company, users often use flink to access redis, hbase or other rpc
services, If some of the Operators are slow to return requests (for external
service reasons), then because Rebalance/Rescale are Round-Robin the Channel
selection policy, so the job is easy to backpressure.
Because the Rebalance/Rescale policy does not care which subtask the data is
sent to downstream, so we expect Rebalance/Rescale to refer to the processing
power of the downstream subtask when choosing a Channel.
Send more data to the free subtask, this ensures the best possible throughput
of job!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
