Hi Weijie, Thanks for the FLIP! I have a few questions regarding the FLIP.
1. +1 to only use XXXParititionStream if users only need to use the configurable PartitionStream. If there are use cases for both, perhaps we could use `ProcessConfigurableNonKeyedPartitionStream` or `ConfigurableNonKeyedPartitionStream` for simplicity. 2. Should we allow users to set custom configurations through the `ProcessConfigurable` interface and access these configurations in the `ProcessFunction` via `RuntimeContext`? I believe it would be useful for process function developers to be able to define custom configurations. 3. How can users define custom metrics within the `ProcessFunction`? Will there be a method like `getMetricGroup` available in the `RuntimeContext`? Best, Xuannan On Fri, Jan 26, 2024 at 2:38 PM Yunfeng Zhou <flink.zhouyunf...@gmail.com> wrote: > > Hi Weijie, > > Thanks for introducing this FLIP! I have a few questions about the > designs proposed. > > 1. Would it be better to have all XXXPartitionStream classes implement > ProcessConfigurable, instead of defining both XXXPartitionStream and > ProcessConfigurableAndXXXPartitionStream? I wonder whether users would > need to operate on a non-configurable PartitionStream. > > 2. The name "ProcessConfigurable" seems a little ambiguous to me. Will > there be classes other than XXXPartitionStream that implement this > interface? Will "Process" be accurate enough to describe > PartitionStream and those classes? > > 3. Apart from the detailed withConfigFoo(foo)/withConfigBar(bar) > methods, would it be better to also add a general > withConfig(configKey, configValue) method to the ProcessConfigurable > interface? Adding a method for each configuration might harm the > readability and compatibility of configurations. > > Looking forward to your response. > > Best regards, > Yunfeng Zhou > > On Tue, Dec 26, 2023 at 2:47 PM weijie guo <guoweijieres...@gmail.com> wrote: > > > > Hi devs, > > > > > > I'd like to start a discussion about FLIP-410: Config, Context and > > Processing Timer Service of DataStream API V2 [1]. This is the second > > sub-FLIP of DataStream API V2. > > > > > > In FLIP-409 [2], we have defined the most basic primitive of > > DataStream V2. On this basis, this FLIP will further answer several > > important questions closely related to it: > > > > 1. > > How to configure the processing over the datastreams, such as > > setting the parallelism. > > 2. > > How to get access to the runtime contextual information and > > services from inside the process functions. > > 3. How to work with processing-time timers. > > > > You can find more details in this FLIP. Its relationship with other > > sub-FLIPs can be found in the umbrella FLIP > > [3]. > > > > > > Looking forward to hearing from you, thanks! > > > > > > Best regards, > > > > Weijie > > > > > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-410%3A++Config%2C+Context+and+Processing+Timer+Service+of+DataStream+API+V2 > > > > [2] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-409%3A+DataStream+V2+Building+Blocks%3A+DataStream%2C+Partitioning+and+ProcessFunction > > > > [3] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-408%3A+%5BUmbrella%5D+Introduce+DataStream+API+V2
