wuchong commented on pull request #14727: URL: https://github.com/apache/flink/pull/14727#issuecomment-770764613
Hi @shouweikun , I have went through the pull request. However, supporting sink parallelism for Hive and Filesytem is not just changing parallelism of the writer DataStream. We should first support `ParallelismProvider` for `DataStreamSinkProvider` first. Because if the sink parallelism is different than the upstream operator, we should implicitly add a keyby shuffle if there is changelog in the stream, otherwise the changelog will be out of order. See https://github.com/apache/flink/blob/95257a255f0da0a95b31647c6d057914d5748871/flink-table/flink-table-planner-blink/src/main/java/org/apache/flink/table/planner/plan/nodes/exec/common/CommonExecSink.java#L116 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
