To the Apache Flink Community Members, I hope this email finds you well. I am writing to discuss a potential improvement for the implementation of a custom commit policy in Flink.
Background: We have encountered challenges in utilizing a custom commit policy due to the inability to pass parameters. This limitation restricts our ability to add additional functionality to the commit policy, such as monitoring the files associated with each commit. Purpose: The purpose of this improvement is to allow the passing of parameters to the custom PartitionCommitPolicy. By enabling parameter passing, users can extend the functionality of their custom commit policy. Example: Suppose we have a custom commit policy called "MyPolicy" that requires parameters such as "key" and "url" for proper functionality. Currently, it is not possible to pass these parameters when using a custom commit policy. However, by introducing the concept of SINK_PARTITION_COMMIT_POLICY_CLASS_PARAMETERS, users can now pass parameters in the following way: By adding SINK_PARTITION_COMMIT_POLICY_CLASS_PARAMETERS, you can pass parameters when using a custom commit policy, for example 'sink.partition-commit.policy.kind'='custom', 'sink.partition-commit.policy.class’='MyPolicy', 'sink.partition-commit.policy.class.parameters’=‘key;url' Eeffect: By adding PartitionCommitPolicyFactory constructor, to ensure backward compatibility for existing user programs. Code PR: To support this improvement, I have submitted a pull request with the necessary code changes. You can find the details of the pull request at the following link: https://github.com/apache/flink/pull/22831/files Best regards, Bangui Dunn
