Apply for confluence Wiki create and edit permission

2019-07-23 Thread Xuannan Su
Hi, I’d like to make a FLIP of the pluggable intermediate result storage, phase 2 of FLIP-36[1]. Could you please give me the create and edit permission for this page[2]. My Id: Xuannan Su Best, Xuannan [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-36%3A+Support+Interactive

[DISCUSS] FLIP-48: Pluggable Intermediate Result Storage

2019-07-24 Thread Xuannan Su
Hi folks, I would like to start the FLIP discussion thread about the pluggable intermediate result storage. This is phase 2 of FLIP-36: Support Interactive Programming in Flink Skip to end of metadata. While the FLIP-36 provides a default implementation of the intermediate result storage using

[DISCUSS] FLIP-326: Enhance Watermark to Support Processing-Time Temporal Join

2023-06-25 Thread Xuannan Su
Hi all, Dong(cc'ed) and I are opening this thread to discuss our proposal to enhance the watermark to properly support processing-time temporal join, which has been documented in FLIP-326 [1]. We want to support the use case where the records from the probe side of the processing-time temporal jo

[DISCUSS] FLIP-329: Add operator attribute to specify support for object-reuse

2023-07-02 Thread Xuannan Su
Hi all, Dong(cc'ed) and I are opening this thread to discuss our proposal to add operator attribute to allow operator to specify support for object-reuse [1]. Currently, the default configuration for pipeline.object-reuse is set to false to avoid data corruption, which can result in suboptimal pe

Re: [DISCUSS] FLIP-329: Add operator attribute to specify support for object-reuse

2023-07-05 Thread Xuannan Su
logic: ture: force object > reusing; false: force deep-copying; unknown: depends on > pipeline.object-reuse config. > > > Best regards, > Jing > > > [1] https://en.wikipedia.org/wiki/JavaBeans > > On Mon, Jul 3, 2023 at 4:25 AM Xuannan Su wrote: > > >

[DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-08-03 Thread Xuannan Su
Hi all, I am opening this thread to discuss FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag[1]. We had a several discussions with Dong Ling about the design, and thanks for all the valuable advice. The FLIP aims to target the use-case where user want to

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-08-13 Thread Xuannan Su
e watermark for > emitting the record attribute? Why not use timestamps in the records? I > don't see any concern in using watermarks. Just wondering if there's any > deep considerations behind this. > > Best, > > Xintong > > > > On Thu, Aug 3, 2023 at 3:0

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-08-14 Thread Xuannan Su
troducing the configuration as is, and changing it later, if needed, with > a regular deprecation and migration process. Just making my suggestions. > > > Best, > > Xintong > > > > On Mon, Aug 14, 2023 at 12:00 PM Xuannan Su wrote: > > > Hi Xintong, > >

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-08-21 Thread Xuannan Su
dize+Connector+Metrics> > > > > On Tue, 15 Aug 2023 at 12:04, Xintong Song <mailto:tonysong...@gmail.com>> wrote: > >> Sounds good to me. >> >> It is true that, if we are introducing the generalized watermark, there >> will be othe

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-08-29 Thread Xuannan Su
ulated exactly? E.g. Watermark > lag = A - B with A is the timestamp of the watermark emitted to the > downstream and B is(this is the part I am not really sure after reading > the FLIP). > > Best regards, > Jing > > > On Mon, Aug 21, 2023 at 9:03 AM Xuannan Su wrote

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-08-30 Thread Xuannan Su
gards, > Jing > > [1] > https://github.com/apache/flink/blob/2c50b4e956305426f478b726d4de4a640a16b810/flink-core/src/main/java/org/apache/flink/api/common/eventtime/WatermarkStrategy.java#L236 > > On Wed, Aug 30, 2023 at 4:06 AM Xuannan Su wrote: > >> Hi Jing, >>

Re: [DISCUSS] FLIP-327: Support stream-batch unified operator to improve job throughput when processing backlog data

2023-08-31 Thread Xuannan Su
Hi all, I would like to share some updates on FLIP-327. Dong and I have had a series of discussions and have made several refinements to the FLIP. The major change to the FLIP is to allow the input of the one-input operator to be automatically sorted during backlog processing. When combined with

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-08-31 Thread Xuannan Su
ort isProcessingBacklog: 1. > From the source; 2. Judged by the watermark lag. What is the priority > between them? > For example, what is the status isProcessingBacklog when the source report > `isProcessingBacklog=false` and the watermark lag exceeds the threshold? > > Best, >

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-09-06 Thread Xuannan Su
de. All data before that range will > be considered as backlog and needed to be processed in the batch mode, > like, e.g. the Present Perfect Progressive tense used in English language. > > Best regards, > Jing > > On Thu, Aug 31, 2023 at 4:45 AM Xuannan Su wrote: > > >

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-09-06 Thread Xuannan Su
> > I thought FLIP-328 will compete with FLIP-309 while setting the value of > the backlog. Understood. Thanks for the hint. > > Best regards, > Jing > > On Wed, Sep 6, 2023 at 12:12 PM Xuannan Su wrote: > > > Hi Jing, > > > > Thank you for the clari

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-09-08 Thread Xuannan Su
hould be “changelog stage” > > I'm not sure if it's enough to support this feature only in FLIP-27 Source. > Although we are pushing the sourceFunction API to be removed, these APIs will > be survive one or two versions in flink repo before they are actually removed. >

Re: [DISCUSS] FLIP-329: Add operator attribute to specify support for object-reuse

2023-09-26 Thread Xuannan Su
e being given to the next operator in the chain.* > >>>>> > >>>>> > >>>>> If I understand you correctly, the hard coding objectReusedCompliant > >>>>> should have higher priority over the configuration, the checking logic > &

Re: [DISCUSS] FLIP-329: Add operator attribute to specify support for object-reuse

2023-10-07 Thread Xuannan Su
tion > is for backward compatibility. > > Best regards, > > Martijn > > On Tue, Sep 26, 2023 at 1:32 AM Xuannan Su wrote: > > > > Hi all, > > > > We would like to revive the discussion and provide a quick update on > > the recent work of the FLIP. We

Re: [ANNOUNCE] New Apache Flink Committer - Jane Chan

2023-10-15 Thread Xuannan Su
Congratulations Jane! Best, Xuannan On Mon, Oct 16, 2023 at 10:21 AM Yun Tang wrote: > > Congratulations, Jane! > > Best > Yun Tang > > From: Rui Fan <1996fan...@gmail.com> > Sent: Monday, October 16, 2023 10:16 > To: dev@flink.apache.org > Cc: qingyue@gmail

Re: [DISCUSS] FLIP-329: Add operator attribute to specify support for object-reuse

2023-10-16 Thread Xuannan Su
Hi Martijn, Do you have further comments regarding the FLIP? If not, I'd like to move forward and start the voting in two days. Best regards, Xuannan On Sat, Oct 7, 2023 at 3:02 PM Xuannan Su wrote: > > Hi Martijn, > > Sorry for the late reply. I don't think it is feasi

[VOTE] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-10-17 Thread Xuannan Su
Hi all, We would like to start the vote for FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag [1]. This FLIP was discussed in this thread [2]. The vote will be open until at least Oct 21st (at least 72 hours), following the consensus voting process. Cheers,

[VOTE] FLIP-329: Add operator attribute to specify support for object-reuse

2023-10-18 Thread Xuannan Su
Hi all, We would like to start the vote for FLIP-329: Add operator attribute to specify support for object-reuse[1]. This FLIP was discussed in this thread [2]. The vote will be open until at least Oct 22nd (at least 72 hours), following the consensus voting process. Cheers, Xuannan [1] https:/

Re: [VOTE] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-10-19 Thread Xuannan Su
Hi all, There is still a discussion ongoing. I will cancel the vote for now. Best Regards, Xuannan On Thu, Oct 19, 2023 at 2:08 PM Dong Lin wrote: > > Thanks for the FLIP! > > +1 (binding) > > On Wed, Oct 18, 2023 at 10:25 AM Xuannan Su wrote: > > > Hi all, > &g

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-13 Thread Xuannan Su
Hi Rui, We are currently revisiting all the configurations for Flink 2.0, and it turns out that many string-based configurations in `ConfigConstants` are deprecated and have been replaced by `ConfigOptions`. Since `ConfigOptions` offers many advantages over string-based configurations for the end

Re: [VOTE] FLIP-396: Trial to test GitHub Actions as an alternative for Flink's current Azure CI infrastructure

2023-12-13 Thread Xuannan Su
Thanks for driving this! +1(non-binding) Best, Xuannan On Thu, Dec 14, 2023 at 11:04 AM Xintong Song wrote: > > +1 (binding) > > Best, > > Xintong > > > > On Thu, Dec 14, 2023 at 10:05 AM Jiabao Sun > wrote: > > > Thanks Matthias for this hard work! > > > > +1(non-binding) > > > > Best, > > Ji

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-19 Thread Xuannan Su
erialized to a json string, and we can only write/read them at once. > The CACHE_FILE_BLOB_KEY is special, its type is byte[], we need to > store it by the setBytes/getBytes. > > Also, I have an offline discussion

Re: [DISCUSS] FLIP-405: Migrate string configuration key to ConfigOption

2023-12-26 Thread Xuannan Su
Hi Xintong and Rui, Thanks for the quick feedback and the suggestions. > 1. I think the default value for `TASK_MANAGER_LOG_PATH_KEY` should be "no > default". I have considered both ways of describing the default value. However, I found out that some of the configurations, such as `web.tmpdir`,

Re: [DISCUSS] FLIP-406: Reorganize State & Checkpointing & Recovery Configuration

2023-12-28 Thread Xuannan Su
Hi Zakelly, Thanks for driving this! The organization of the configuration option in the FLIP looks much cleaner and easier to understand. +1 to the FLIP. Just some questions from me. 1. I think the change to the ConfigOptions should be put in the `Public Interface` section, instead of `Proposed

Re: [DISCUSS] FLIP-405: Migrate string configuration key to ConfigOption

2024-01-01 Thread Xuannan Su
> > On Wed, Dec 27, 2023 at 1:56 PM Hang Ruan wrote: >> >> Hi, Rui Fan. >> >> Thanks for this FLIP. >> >> I think the key of LOCAL_NUMBER_TASK_MANAGER is better as >> 'minicluster.number-of-taskmanagers' or 'minicluster.taskmanager-number' >

Re: [DISCUSS] FLIP-329: Add operator attribute to specify support for object-reuse

2024-01-05 Thread Xuannan Su
> I was interested in getting clarification as to what you meant by “or >>> access references…”, to see if it covered this situation: >>> >>> StreamX —forward--> operator1 >>> StreamX —forward--> operator2 >>> >>> If operator1 mod

Re: [DISCUSS] FLIP-329: Add operator attribute to specify support for object-reuse

2024-01-07 Thread Xuannan Su
gt; appreciated! I realize there is already a voting thread "[VOTE] FLIP-329: Add > operator attribute to specify support for object-reuse". What else do we need? > > Best > Lu > > On Fri, Jan 5, 2024 at 12:46 AM Xuannan Su wrote: >> >> Hi Lu, >> >&g

Re: [DISCUSS] FLIP-405: Migrate string configuration key to ConfigOption

2024-01-07 Thread Xuannan Su
Hi all, Thanks for the discussion. I think all the comments and questions have been addressed. I will open the voting thread today. Best, Xuannan On Tue, Jan 2, 2024 at 11:59 AM Xuannan Su wrote: > > Hi all, > > Thank you for all your comments! The FLIP has been updated > acco

[VOTE] FLIP-405: Migrate string configuration key to ConfigOption

2024-01-07 Thread Xuannan Su
Hi everyone, Thanks for all the feedback about the FLIP-405: Migrate string configuration key to ConfigOption [1] [2]. I'd like to start a vote for it. The vote will be open for at least 72 hours(excluding weekends,until Jan 11, 12:00AM GMT) unless there is an objection or an insufficient number

Re: [DISCUSS] FLIP-329: Add operator attribute to specify support for object-reuse

2024-01-09 Thread Xuannan Su
Is the requirement to send an email thread about the voting? > What else is needed? What's the passing criteria? > > Best > Lu > > On Sun, Jan 7, 2024 at 5:41 PM Xuannan Su wrote: >> >> Hi Liu, >> >> The voting thread has been open for a long ti

Re: Re: [DISCUSS] FLIP-331: Support EndOfStreamWindows and isOutputOnEOF operator attribute to optimize task deployment

2024-01-10 Thread Xuannan Su
Hi all, After several rounds of offline discussions with Xingtong and Jinhao, we have decided to narrow the scope of the FLIP. It will now focus on introducing OperatorAttributes that indicate whether an operator emits records only after inputs have ended. We will also use the attribute to optimiz

Re: [DISCUSS] FLIP-406: Reorganize State & Checkpointing & Recovery Configuration

2024-01-10 Thread Xuannan Su
akelly, > > >> > > > > > > > >> > > > > > I'm not sure whether we could add the state backend type in > > the > > >> > > > > > new key name of state.backend.incremental. It means we use > > >> > > > >

Re: Re: Re: [VOTE] Accept Flink CDC into Apache Flink

2024-01-10 Thread Xuannan Su
+1 (non-binding) Best, Xuannan On Thu, Jan 11, 2024 at 10:28 AM Xuyang wrote: > > +1 (non-binding)-- > > Best! > Xuyang > > > > > > 在 2024-01-11 10:00:11,"Yang Wang" 写道: > >+1 (binding) > > > > > >Best, > >Yang > > > >On Thu, Jan 11, 2024 at 9:53 AM liu ron wrote: > > > >> +1 non-bindi

Re: [VOTE] FLIP-405: Migrate string configuration key to ConfigOption

2024-01-11 Thread Xuannan Su
g Ruan wrote: > > > +1(non-binding) > > > > Best, > > Hang > > > > Rui Fan <1996fan...@gmail.com> 于2024年1月8日周一 13:04写道: > > > > > +1(binding) > > > > > > Best, > > > Rui > > > > > > On Mon, Jan 8,

Re: [VOTE] FLIP-405: Migrate string configuration key to ConfigOption

2024-01-11 Thread Xuannan Su
, 2024 at 1:36 PM Xuannan Su wrote: > > Hi all, > > During voting, we identified two improvements we'd like to make to the FLIP: > > - We will mark the getBytes(String key, byte[] defaultValue) and > setBytes(String key, byte[] bytes) methods as @Internal, as they are > i

[DISCUSS] FLIP-36 - Support Interactive Programming in Flink Table API

2020-04-22 Thread Xuannan Su
Hi folks, I'd like to start the discussion about FLIP-36 Support Interactive Programming in Flink Table API https://cwiki.apache.org/confluence/display/FLINK/FLIP-36%3A+Support+Interactive+Programming+in+Flink The FLIP proposes to add support for interactive programming in Flink Table API. Specif

Re: [DISCUSS] FLIP-36 - Support Interactive Programming in Flink Table API

2020-04-24 Thread Xuannan Su
when auto caching is enabled will be discussed in detail by a new FLIP. I will remove the descriptor to avoid further confusion. > Thanks, > > Jiangjie (Becket) Qin > > > > On Wed, Apr 22, 2020 at 4:00 PM Xuannan Su wrote: > > > Hi folks, > > > &g

Re: [DISCUSS] FLIP-36 - Support Interactive Programming in Flink Table API

2020-04-24 Thread Xuannan Su
> re-generated if it is gone? > > Thanks, > > Jiangjie (Becket) Qin > > > On Fri, Apr 24, 2020 at 3:59 PM Xuannan Su wrote: > > > Hi Becket, > > > > Thanks for the comments. > > > > On Fri, Apr 24, 2020 at 9:12 AM Becket Qin wrote: > > >

Re: [DISCUSS] FLIP-36 - Support Interactive Programming in Flink Table API

2020-04-25 Thread Xuannan Su
nk? > > Thanks, > > Jiangjie (Becket) Qin > > On Fri, Apr 24, 2020 at 5:00 PM Xuannan Su wrote: > > > Hi Becket, > > > > The intermediate result will indeed be automatically re-generated by > > resubmitting the original DAG. And that job could fail

Re: [ANNOUNCE] Becket Qin joins the Flink PMC

2019-10-29 Thread Xuannan Su
Congratulations Becket! :-) Xuannan > On Oct 28, 2019, at 6:07 AM, Fabian Hueske wrote: > > Hi everyone, > > I'm happy to announce that Becket Qin has joined the Flink PMC. > Let's congratulate and welcome Becket as a new member of the Flink PMC! > > Cheers, > Fabian

Re: [DISCUSS] FLIP-36 - Support Interactive Programming in Flink Table API

2020-09-10 Thread Xuannan Su
It should explain how > this topic is split into the subtasks and potential PRs. Ideally, those > subtasks are split by layer to be reviewed by the individual component > teams. > > Thanks, > Timo > > > > On 25.08.20 12:45, Xuannan Su wrote: > > Hi T

Re: [DISCUSS] FLIP-36 - Support Interactive Programming in Flink Table API

2020-09-14 Thread Xuannan Su
:00, Xuannan Su wrote: > > > How do you imagine that? Where do you distinguish between per-job and > > > session mode? > > The StreamExecutionEnvironment can distinguish between per-job and session > > mode by the type of the PipelineExecutor, i.e, A

Re: [DISCUSS] FLIP-36 - Support Interactive Programming in Flink Table API

2020-09-15 Thread Xuannan Su
On Sep 15, 2020, 3:28 PM +0800, Aljoscha Krettek , wrote: > On 15.09.20 07:00, Xuannan Su wrote: > > Thanks for your comment. I agree that we should not introduce tight > > coupling with PipelineExecutor to the execution environment. With that in > > mind, to distinguish th

Re: [DISCUSS] FLIP-36 - Support Interactive Programming in Flink Table API

2020-09-15 Thread Xuannan Su
, wrote: > On 15.09.20 10:54, Xuannan Su wrote: > > One way of solving this is to let the CatalogManager probe the existence of > > the IntermediateResult so that the planner can decide if the cache table > > should be used. > > That could be a reasonable solution, yes. > > Best, > Aljoscha

[VOTE] FLIP-36 - Support Interactive Programming in Flink Table API

2020-09-15 Thread Xuannan Su
Hi all, I'd like to start the vote for FLIP-36[1], which has been discussed in thread[2]. The vote will be open for 72h, until September 19, 2020, 04:00 AM UTC, unless there's an objection or not enough votes. Best, Xuannan [1]  https://cwiki.apache.org/confluence/display/FLINK/FLIP-36%3A+Sup

[RESULT][VOTE] FLIP-36 - Support Interactive Programming in Flink Table API

2020-09-20 Thread Xuannan Su
Hi all, The voting time for FLIP-36 has passed. I'm closing the vote now. There were 3 binding votes: - Aljoscha (binding) - Timo (binding) - Becket (binding) There were no disapproving votes. Thus, FLIP-36 has been accepted. Thanks everyone for joining the discussion and giving feedback! Bes

Re: [DISCUSS] FLIP-416: Deprecate and remove the RestoreMode#LEGACY

2024-01-14 Thread Xuannan Su
Hi Zakelly, Thanks for driving this. +1 to removing the LEGACY mode. Best regards, Xuannan On Mon, Jan 15, 2024 at 3:22 AM Danny Cranmer wrote: > > +1 to removing LEGACY mode in Flink 2.0. Thanks for driving. > > Danny, > > On Sat, 13 Jan 2024, 08:20 Yanfei Lei, wrote: > > > Thanks Zakelly for

Re: [VOTE] FLIP-405: Migrate string configuration key to ConfigOption

2024-01-14 Thread Xuannan Su
Hi all, Thank you all! Closing the vote. The result will be announced in a separate email. Best regards, Xuannan On Fri, Jan 12, 2024 at 2:44 PM Zhu Zhu wrote: > > +1 (binding) > > Thanks, > Zhu > > Xuannan Su 于2024年1月12日周五 14:24写道: > > > Hi all, > > >

[RESULT][VOTE] FLIP-405: Migrate string configuration key to ConfigOption

2024-01-14 Thread Xuannan Su
Hi devs, I'm happy to announce that FLIP-405: Migrate string configuration key to ConfigOption [1] has been accepted with 4 approving votes (3 binding) [2]: - Rui Fan (binding) - Hang Ruan (non-binding) - Xintong Song (binding) - Zhu Zhu (binding) There are no disapproving votes. Thanks again t

Re: [VOTE] FLIP-406: Reorganize State & Checkpointing & Recovery Configuration

2024-01-24 Thread Xuannan Su
+1 (non-binding) Best, Xuannan On Thu, Jan 25, 2024 at 10:15 AM Lijie Wang wrote: > > +1 (binding) > > Best, > Lijie > > Yanfei Lei 于2024年1月25日周四 10:06写道: > > > +1 (binding) > > > > Hangxiang Yu 于2024年1月25日周四 10:00写道: > > > > > > +1 (binding) > > > > > > On Thu, Jan 25, 2024 at 8:49 AM Rui Fan

Re: [DISCUSS] FLIP-408: [Umbrella] Introduce DataStream API V2

2024-01-29 Thread Xuannan Su
Hi Weijie, Thanks for driving the work! There are indeed many pain points in the current DataStream API, which are challenging to resolve with its existing design. It is a great opportunity to propose a new DataStream API that tackles these issues. I like the way we've divided the FLIP into multip

Re: [DISCUSS] FLIP-409: DataStream V2 Building Blocks: DataStream, Partitioning and ProcessFunction

2024-01-29 Thread Xuannan Su
Hi Weijie, Thank you for driving the design of the new DataStream API. I have a few questions regarding the FLIP: 1. In the partitioning section, it says that "broadcast can only be used as a side-input of other Inputs." Could you clarify what is meant by "side-input"? If I understand correctly,

Re: [DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-01-29 Thread Xuannan Su
Hi Weijie, Thanks for the FLIP! I have a few questions regarding the FLIP. 1. +1 to only use XXXParititionStream if users only need to use the configurable PartitionStream. If there are use cases for both, perhaps we could use `ProcessConfigurableNonKeyedPartitionStream` or `ConfigurableNonKeyed

Re: Re: [DISCUSS] FLIP-331: Support EndOfStreamWindows and isOutputOnEOF operator attribute to optimize task deployment

2024-01-29 Thread Xuannan Su
Hi all, Thanks for the comments and suggestions. If there are no further comments, I will open the voting thread tomorrow. Best regards, Xuannan On Fri, Jan 26, 2024 at 3:16 PM Dong Lin wrote: > > Thanks Xuannan for the update! > > +1 (binding) > > On Wed, Jan 10, 2024 at

[VOTE] FLIP-331: Support EndOfStreamTrigger and isOutputOnlyAfterEndOfStream operator attribute to optimize task deployment

2024-01-30 Thread Xuannan Su
Hi everyone, Thanks for all the feedback about the FLIP-331: Support EndOfStreamTrigger and isOutputOnlyAfterEndOfStream operator attribute to optimize task deployment [1] [2]. I'd like to start a vote for it. The vote will be open for at least 72 hours(excluding weekends,until Feb 5, 12:00AM GMT

Re: [DISCUSS] FLIP-409: DataStream V2 Building Blocks: DataStream, Partitioning and ProcessFunction

2024-01-31 Thread Xuannan Su
gt; > > > > > > > I actually also considered this way at first, but it would have to > > > > introduce some concepts like ConnectedStreams. But we hope that streams > > > > will be more clearly defined in the DataStream API, otherwise we will > &

Re: [VOTE] FLIP-331: Support EndOfStreamTrigger and isOutputOnlyAfterEndOfStream operator attribute to optimize task deployment

2024-02-05 Thread Xuannan Su
n-binding) > > > > Best, > > Hang > > > > Dong Lin 于2024年2月5日周一 11:08写道: > > > > > Thanks for the FLIP. > > > > > > +1 (binding) > > > > > > Best, > > > Dong > > > > > > On Wed, Jan 3

[RESULT][VOTE] FLIP-331: Support EndOfStreamTrigger and isOutputOnlyAfterEndOfStream operator attribute to optimize task deployment

2024-02-05 Thread Xuannan Su
Hi devs, I'm happy to announce that FLIP-331: Support EndOfStreamTrigger and isOutputOnlyAfterEndOfStream operator attribute to optimize task deployment [1] has been accepted with 6 approving votes (4 binding) [2]: - Xintong Song (binding) - Rui Fan (binding) - Weijie Guo (binding) - Dong Lin (bi

Re: [VOTE] FLIP-408: [Umbrella] Introduce DataStream API V2

2024-02-26 Thread Xuannan Su
+1 (non-binding) Best, Xuannan On Tue, Feb 27, 2024 at 9:36 AM Xintong Song wrote: > > +1 (binding) > > Best, > > Xintong > > > > On Mon, Feb 26, 2024 at 6:08 PM weijie guo > wrote: > > > Hi everyone, > > > > > > Thanks for all the feedback about the FLIP-408: [Umbrella] Introduce > > DataStre

Re: [VOTE] FLIP-409: DataStream V2 Building Blocks: DataStream, Partitioning and ProcessFunction

2024-02-26 Thread Xuannan Su
+1 (non-binding) Best, Xuannan On Tue, Feb 27, 2024 at 9:38 AM Xintong Song wrote: > > +1 (binding) > > Best, > > Xintong > > > > On Mon, Feb 26, 2024 at 6:09 PM weijie guo > wrote: > > > Hi everyone, > > > > > > Thanks for all the feedback about the FLIP-409: DataStream V2 Building > > Blocks:

Re: [VOTE] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-02-26 Thread Xuannan Su
+1 (non-binding) Best, Xuannan On Tue, Feb 27, 2024 at 9:37 AM Xintong Song wrote: > > +1 (binding) > > Best, > > Xintong > > > > On Mon, Feb 26, 2024 at 6:10 PM weijie guo > wrote: > > > Hi everyone, > > > > > > Thanks for all the feedback about the FLIP-410: Config, Context and > > Processing

Re: [DISCUSS] FLIP-424: Asynchronous State APIs

2024-03-11 Thread Xuannan Su
Hi Zakelly, Thank you for the proposal! Please find my comments below. 1. The name `emptyFuture` seems a little unintuitive, and it is hard to understand in what use case the `emptyFuture` should be used. If I understand correctly, it is similar to the FutureUtils#completedVoidFuture. How about n

Re: [DISCUSS] FLIP-424: Asynchronous State APIs

2024-03-11 Thread Xuannan Su
Hi Zakelly, Thanks for the quick response. > It will be used in callback chaining cases where some branch within one > callback does nothing. I'm in favor of short phrases to express the > functionalities. Thus I suggest `completedVoidFuture` or `voidFuture`, WDTY? I'd prefer `completedVoidFutur

Re: [VOTE] Release 1.19.0, release candidate #2

2024-03-12 Thread Xuannan Su
+1 (non-binding) - Verified signature and checksum - Verified that source distribution does not contain binaries - Built from source code successfully - Reviewed the release announcement PR Best regards, Xuannan On Tue, Mar 12, 2024 at 2:18 PM Hang Ruan wrote: > > +1 (non-binding) > > - Verifie

Re: [ANNOUNCE] Apache Flink 1.19.0 released

2024-03-18 Thread Xuannan Su
Congratulations! Thanks for all the great work! Best regards, Xuannan On Tue, Mar 19, 2024 at 1:31 PM Yu Li wrote: > > Congrats and thanks all for the efforts! > > Best Regards, > Yu > > On Tue, 19 Mar 2024 at 11:51, gongzhongqiang > wrote: > > > > Congrats! Thanks to everyone involved! > > >

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-21 Thread Xuannan Su
Congratulations! Best regards, Xuannan On Fri, Mar 22, 2024 at 9:17 AM Charles Zhang wrote: > > Congratulations! > > Best wishes, > Charles Zhang > from Apache InLong > > > Jeyhun Karimov 于2024年3月22日周五 04:16写道: > > > Great news! Congratulations! > > > > Regards, > > Jeyhun > > > > On Thu, Mar 2

Re: [VOTE] FLIP-424: Asynchronous State APIs

2024-03-28 Thread Xuannan Su
+1 (non-binding) Best, Xuannan On Wed, Mar 27, 2024 at 6:23 PM Zakelly Lan wrote: > > Hi devs, > > I'd like to start a vote on the FLIP-424: Asynchronous State APIs [1]. The > discussion thread is here [2]. > > The vote will be open for at least 72 hours unless there is an objection or > insuff

Re: [VOTE] FLIP-425: Asynchronous Execution Model

2024-03-28 Thread Xuannan Su
+1 (non-binding) Best regards, Xuannan On Wed, Mar 27, 2024 at 6:28 PM Yanfei Lei wrote: > > Hi everyone, > > Thanks for all the feedback about the FLIP-425: Asynchronous Execution > Model [1]. The discussion thread is here [2]. > > The vote will be open for at least 72 hours unless there is an

[DISCUSSION] FLIP-442: FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-09 Thread Xuannan Su
Hi all, I'd like to start a discussion on FLIP-442: General Improvement to Configuration for Flink 2.0 [1]. As Flink moves toward 2.0, we aim to provide users with a better experience with the existing configuration. This FLIP proposes several general improvements to the current configuration. Lo

Re: [DISCUSSION] FLIP-442: FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-10 Thread Xuannan Su
tated > with @PublicEvolving. > > > Best, > Zakelly > > > On Tue, Apr 9, 2024 at 4:21 PM Xuannan Su wrote: > > > Hi all, > > > > I'd like to start a discussion on FLIP-442: General Improvement to > > Configuration for Flink 2.0 [1]. As Flink mo

Re: [DISCUSSION] FLIP-442: FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-10 Thread Xuannan Su
> We will relocate these ConfigOptions to a class that is included > > in the documentation generation. > > Would it make sense to define also in the FLIP the options class for > these variables? For example, GPUDriverOptions? > > Best, > Muhammet > > On 2024-04-09

Re: [DISCUSSION] FLIP-442: FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-14 Thread Xuannan Su
gt; > Regarding the new classes, we propose updating the > > `ConfigOptionsDocGenerator` to throw an exception and fail the build > > if it detects any new class missing the proper annotation in the > > future. > > Thanks for your feedback, it sounds good to me. > > Be

Re: [ANNOUNCE] New Apache Flink PMC Member - Lincoln Lee

2024-04-15 Thread Xuannan Su
Congratulations, Lincoln! Best regards, Xuannan On Tue, Apr 16, 2024 at 10:04 AM Hang Ruan wrote: > > Congratulations, Lincoln! > > Best, > Hang > > yh z 于2024年4月16日周二 09:14写道: > > > Congratulations, Lincoln! > > > > Best, > > Yunhong (Swuferhong) > > > > > > Swapnal Varma 于2024年4月15日周一 18:50写

Re: [ANNOUNCE] New Apache Flink Committer - Zakelly Lan

2024-04-15 Thread Xuannan Su
Congratulations Zakelly! Best regards, Xuannan On Mon, Apr 15, 2024 at 4:31 PM Jing Ge wrote: > > Congratulations Zakelly! > > Best regards, > Jing > > On Mon, Apr 15, 2024 at 4:26 PM Xia Sun wrote: > > > Congratulations Zakelly! > > > > Best, > > Xia > > > > Leonard Xu 于2024年4月15日周一 16:16写道

[VOTE] FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-16 Thread Xuannan Su
Hi everyone, Thanks for all the feedback about the FLIP-442: General Improvement to Configuration for Flink 2.0 [1] [2]. I'd like to start a vote for it. The vote will be open for at least 72 hours(excluding weekends,until APR 22, 12:00AM GMT) unless there is an objection or an insufficient numbe

Re: [VOTE] FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-22 Thread Xuannan Su
Hi all, Thank you all! Closing the vote. The result will be announced in a separate email. Best regards, Xuannan On Fri, Apr 19, 2024 at 10:58 AM gongzhongqiang wrote: > > +1 (non-binding) > > > Best, > Zhongqiang Gong > > Xuannan Su 于2024年4月17日周三 13:02写道: > >

[RESULT][VOTE] FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-22 Thread Xuannan Su
Hi devs, I'm happy to announce that FLIP-442: General Improvement to Configuration for Flink 2.0[1] has been accepted with 9 approving votes (4 bindings) [2]: - Rui Fan (binding) - Zakelly Lan (binding) - Xintong Song (binding) - Yuxin Tan (non-binding) - Zhu Zhu (binding) - Jeyhun Karimov (non-b

[DISCUSSION] FLIP-450: Improve Runtime Configuration for Flink 2.0

2024-04-25 Thread Xuannan Su
Hi all, I'd like to start a discussion on FLIP-450: Improve Runtime Configuration for Flink 2.0 [1]. As Flink moves toward 2.0, we have revisited all runtime configurations and identified several improvements to enhance user-friendliness and maintainability. In this FLIP, we aim to refine the runt

Re: [DISCUSSION] FLIP-450: Improve Runtime Configuration for Flink 2.0

2024-05-10 Thread Xuannan Su
t might be nicer to provide an implementation plan with a list > of related tasks in the FLIP. This should not block the FLIP though. > > Best, > > Xintong > > > > On Thu, Apr 25, 2024 at 4:35 PM Xuannan Su wrote: > > > Hi all, > > > > I'd like to

Re: [DISCUSSION] FLIP-450: Improve Runtime Configuration for Flink 2.0

2024-05-10 Thread Xuannan Su
locking shuffle and legacy > > > hybrid shuffle mode, and the behavior change of overdraft network > > buffers. > > > Therefore, it might be nicer to provide an implementation plan with a > > list > > > of related tasks in the FLIP. This should not block the F

Re: [DISCUSSION] FLIP-450: Improve Runtime Configuration for Flink 2.0

2024-05-12 Thread Xuannan Su
, +1 for this proposal. > > > > Best, > > Rui > > > > On Sat, May 11, 2024 at 10:20 AM Xuannan Su wrote: > > > > > Hi Rui, > > > > > > Thanks for the suggestion! > > > > > > I updated the description of > > &g

Re: [DISCUSSION] FLIP-450: Improve Runtime Configuration for Flink 2.0

2024-05-13 Thread Xuannan Su
update. The FLIP looks good to me. +1 for it. > > Regards, > Jeyhun > > On Mon, May 13, 2024 at 4:45 AM Xuannan Su wrote: > > > Hi Jeyhun, > > > > Thanks for the comment! > > > > Yes, we intended to remove the StreamPiplineOptions in 2.0. I upda

[VOTE] FLIP-450: Improve Runtime Configuration for Flink 2.0

2024-05-15 Thread Xuannan Su
Hi everyone, Thanks for all the feedback about the FLIP-450: Improve Runtime Configuration for Flink 2.0 [1] [2]. I'd like to start a vote for it. The vote will be open for at least 72 hours(excluding weekends,until MAY 20, 12:00AM GMT) unless there is an objection or an insufficient number of vo

Re: [DISCUSSION] FLIP-457: Improve Table/SQL Configuration for Flink 2.0

2024-05-15 Thread Xuannan Su
Hi Jane, Thanks for driving this effort! And +1 for the proposed changes. I have one comment on the migration plan. For options to be moved to another module/package, I think we have to mark the old option deprecated in 1.20 for it to be removed in 2.0, according to the API compatibility guarant

Re: [DISCUSSION] FLIP-450: Improve Runtime Configuration for Flink 2.0

2024-05-15 Thread Xuannan Su
y are going to be removed for v2, it would be good to > explicitly document the thinking and impact of the deprecations and > consequent removals, > Kind regards, David. > > From: Xuannan Su > Date: Tuesday, 14 May 2024 at 02:08 > To: dev@flink.apache.org > Su

Re: [VOTE] FLIP-450: Improve Runtime Configuration for Flink 2.0

2024-05-20 Thread Xuannan Su
guo > wrote: > > > +1(binding) > > > > Best regards, > > > > Weijie > > > > > > Rui Fan <1996fan...@gmail.com> 于2024年5月15日周三 17:50写道: > > > > > +1(binding) > > > > > > Best, > > > Rui > > > > >

[RESULT][VOTE] FLIP-450: Improve Runtime Configuration for Flink 2.0

2024-05-20 Thread Xuannan Su
Hi devs, I'm happy to announce that FLIP-450: Improve Runtime Configuration for Flink 2.0[1] has been accepted with 3 approving votes (3 bindings) [2]: - Rui Fan (binding) - Weijie Guo (binding) - Xintong Song (binding) There are no disapproving votes. Thanks again to everyone who participated

Re: [VOTE] FLIP-457: Improve Table/SQL Configuration for Flink 2.0

2024-05-27 Thread Xuannan Su
+1 (non-binding) Best regards, Xuannan On Mon, May 27, 2024 at 3:43 PM Jark Wu wrote: > > +1 (binding) > > Best, > Jark > > On Mon, 27 May 2024 at 14:29, Hang Ruan wrote: > > > +1 (non-binding) > > > > Best, > > Hang > > > > gongzhongqiang 于2024年5月27日周一 14:16写道: > > > > > +1 (non-binding) > >

Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo

2024-06-05 Thread Xuannan Su
Congratulations! Best regards, Xuannan On Thu, Jun 6, 2024 at 9:20 AM Jiabao Sun wrote: > > Congratulations, Weijie! > > Best, > Jiabao > > Lincoln Lee 于2024年6月6日周四 09:17写道: > > > Congratulations, Weijie! > > > > Best, > > Lincoln Lee > > > > > > ConradJam 于2024年6月5日周三 10:46写道: > > > > > Congr

Re: [ANNOUNCE] New Apache Flink PMC Member - Fan Rui

2024-06-05 Thread Xuannan Su
Congratulations! Best regards, Xuannan On Thu, Jun 6, 2024 at 9:53 AM Hangxiang Yu wrote: > > Congratulations, Rui ! > > On Thu, Jun 6, 2024 at 9:18 AM Lincoln Lee wrote: > > > Congratulations, Rui! > > > > Best, > > Lincoln Lee > > > > > > Lijie Wang 于2024年6月6日周四 09:11写道: > > > > > Congratula

[DISCUSS] FLIP-205: Support cache in DataStream for Batch Processing

2021-12-29 Thread Xuannan Su
Hi devs, I’d like to start a discussion about adding support to cache the intermediate result at DataStream API for batch processing. As the DataStream API now supports batch execution mode, we see users using the DataStream API to run batch jobs. Interactive programming is an important use case

Re: [DISCUSS] FLIP-205: Support cache in DataStream for Batch Processing

2021-12-29 Thread Xuannan Su
nd to use the higher level APIs, that allow for > faster prototyping (Table API in Flink's case). Should the Table API also > be covered by this FLIP? > > Best, > D. > > On Wed, Dec 29, 2021 at 10:36 AM Xuannan Su wrote: > > > Hi devs, > > > > I’d like

Re: [DISCUSS] FLIP-205: Support cache in DataStream for Batch Processing

2022-01-03 Thread Xuannan Su
replied, caching at Table/SQL API is the next step, as a part of > interactive programming in Table API, which we all agree is the major > scenario. What do you think about the relation between it and FLIP-188? > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-188%3A+In

Re: [DISCUSS] FLIP-205: Support cache in DataStream for Batch Processing

2022-01-04 Thread Xuannan Su
I is the next step, as a part of > > interactive programming in Table API, which we all agree is the major > > scenario. What do you think about the relation between it and FLIP-188? > > > > [1] > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-

Re: [DISCUSS] FLIP-205: Support cache in DataStream for Batch Processing

2022-01-05 Thread Xuannan Su
like further steps in an interactive program. > > > > > > As you replied, caching at Table/SQL API is the next step, as a part of > > > interactive programming in Table API, which we all agree is the major > > > scenario. What do you think about the relation bet

Re: [DISCUSS] FLIP-205: Support cache in DataStream for Batch Processing

2022-01-10 Thread Xuannan Su
> On Wed, Jan 5, 2022 at 11:54 PM Zhipeng Zhang > wrote: > > > Hi Xuannnan, > > > > Thanks for the reply. > > > > Regarding whether and how to support cache sideoutput, I agree that the > > second option might be better if there do exist a use case that users

  1   2   3   >