Re: [DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-02-19 Thread weijie guo
Hi All, Thanks for all the feedback. If there are no more comments, I would like to start the vote thread, thanks again! Best regards, Weijie Xintong Song 于2024年2月20日周二 14:17写道: > LGTM > > Best, > > Xintong > > > > On Mon, Feb 19, 2024 at 10:48 AM weijie guo > wrote: > > > Hi All, > > > >

Re: [DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-02-19 Thread Xintong Song
LGTM Best, Xintong On Mon, Feb 19, 2024 at 10:48 AM weijie guo wrote: > Hi All, > > Based on the discussion thread of FLIP-409, I did a synchronized update to > this one. In simple terms, added `TwoInputBroadcastStreamProcessFunction` > related content. > > > Best regards, > > Weijie > > > w

Re: [DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-02-18 Thread weijie guo
Hi All, Based on the discussion thread of FLIP-409, I did a synchronized update to this one. In simple terms, added `TwoInputBroadcastStreamProcessFunction` related content. Best regards, Weijie weijie guo 于2024年1月31日周三 15:00写道: > Hi Xintong, > > Thanks for the quick reply. > > > Why introd

Re: [DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-01-30 Thread weijie guo
Hi Xintong, Thanks for the quick reply. > Why introduce a new `MetricManager` rather than just return `MetricGroup` from `RuntimeContext`? This is to facilitate possible future extensions. But I thought it through, MetricGroup itself also plays the role of a manager. So I think you are right, I

Re: [DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-01-30 Thread Xintong Song
> > > How can users define custom metrics within the `ProcessFunction`? > Will there be a method like `getMetricGroup` available in the > `RuntimeContext`? > > I think this is a reasonable request. For extensibility, I have added the > getMetricManager instead of getMetricGroup to RuntimeContext, w

Re: [DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-01-29 Thread weijie guo
Hi Wencong, > Q1. In the "Configuration" section, it is mentioned that configurations can be set continuously using the withXXX methods. Are these configuration options the same as those provided by DataStream V1, or might there be different options compared to V1? I haven't considered options th

Re: [DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-01-29 Thread weijie guo
Hi Xuannan, > 1. +1 to only use XXXParititionStream if users only need to use the configurable PartitionStream. If there are use cases for both, perhaps we could use `ProcessConfigurableNonKeyedPartitionStream` or `ConfigurableNonKeyedPartitionStream` for simplicity. As for why we need both, you

Re: [DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-01-29 Thread weijie guo
Thanks Yunfeng, Let me try to answer your question :) > 1. Would it be better to have all XXXPartitionStream classes implement ProcessConfigurable, instead of defining both XXXPartitionStream and ProcessConfigurableAndXXXPartitionStream? I wonder whether users would need to operate on a non-confi

Re: [DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-01-29 Thread Xuannan Su
Hi Weijie, Thanks for the FLIP! I have a few questions regarding the FLIP. 1. +1 to only use XXXParititionStream if users only need to use the configurable PartitionStream. If there are use cases for both, perhaps we could use `ProcessConfigurableNonKeyedPartitionStream` or `ConfigurableNonKeyed

Re: [DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-01-25 Thread Yunfeng Zhou
Hi Weijie, Thanks for introducing this FLIP! I have a few questions about the designs proposed. 1. Would it be better to have all XXXPartitionStream classes implement ProcessConfigurable, instead of defining both XXXPartitionStream and ProcessConfigurableAndXXXPartitionStream? I wonder whether us

Re:[DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-01-24 Thread Wencong Liu
Hi Weijie, Regarding FLIP-410, I have the following questions: Q1. In the "Configuration" section, it is mentioned that configurations can be set continuously using the withXXX methods. Are these configuration options the same as those provided by DataStream V1, or might there be different op

[DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2023-12-25 Thread weijie guo
Hi devs, I'd like to start a discussion about FLIP-410: Config, Context and Processing Timer Service of DataStream API V2 [1]. This is the second sub-FLIP of DataStream API V2. In FLIP-409 [2], we have defined the most basic primitive of DataStream V2. On this basis, this FLIP will further answ