Re: [DISCUSS] FLIP-346: Deprecate ManagedTable related APIs

2023-07-21 Thread Jane Chan
Hi, all, Since there are no objections against the FLIP, I'm preparing to start a vote. Please don't hesitate to follow up on this if you have any concerns. Best, Jane On Fri, Jul 21, 2023 at 10:19 AM Jane Chan wrote: > Hi Leonard, > > I got your point and adjusted the FLIP to cover this. Tha

Re: [DISCUSS][2.0] FLIP-343: Remove parameter in WindowAssigner#getDefaultTrigger()

2023-07-21 Thread Jing Ge
+1 NIT: the release in the FLIP is still empty, it should be 2.0 Best regards, Jing On Fri, Jul 21, 2023 at 6:03 AM Xintong Song wrote: > +1 > > Best, > > Xintong > > > > On Fri, Jul 21, 2023 at 10:53 AM Wencong Liu wrote: > > > Hi devs, > > > > I would like to start a discussion on FLIP-343:

[VOTE] FLIP-346: Deprecate ManagedTable related APIs

2023-07-21 Thread Jane Chan
Hi developers, Thanks for all the feedback on FLIP-346: Deprecate ManagedTable related APIs[1]. Based on the discussion[2], we have reached a consensus, so I would like to start a vote. The vote will last for at least 72 hours unless there is an objection or insufficient votes. [1] https://cwiki

Re: [VOTE] FLIP-346: Deprecate ManagedTable related APIs

2023-07-21 Thread yuxia
+1(binging) Best regards, Yuxia - 原始邮件 - 发件人: "Jane Chan" 收件人: "dev" 发送时间: 星期五, 2023年 7 月 21日 下午 3:41:11 主题: [VOTE] FLIP-346: Deprecate ManagedTable related APIs Hi developers, Thanks for all the feedback on FLIP-346: Deprecate ManagedTable related APIs[1]. Based on the discussion[2],

Re: [VOTE] FLIP-346: Deprecate ManagedTable related APIs

2023-07-21 Thread Leonard Xu
+1 Best, Leonard > On Jul 21, 2023, at 4:02 PM, yuxia wrote: > > +1(binging) > > Best regards, > Yuxia > > - 原始邮件 - > 发件人: "Jane Chan" > 收件人: "dev" > 发送时间: 星期五, 2023年 7 月 21日 下午 3:41:11 > 主题: [VOTE] FLIP-346: Deprecate ManagedTable related APIs > > Hi developers, > > Thanks for a

Re: [VOTE] FLIP-346: Deprecate ManagedTable related APIs

2023-07-21 Thread Lincoln Lee
+1 Best, Lincoln Lee Leonard Xu 于2023年7月21日周五 16:07写道: > +1 > > Best, > Leonard > > > On Jul 21, 2023, at 4:02 PM, yuxia wrote: > > > > +1(binging) > > > > Best regards, > > Yuxia > > > > - 原始邮件 - > > 发件人: "Jane Chan" > > 收件人: "dev" > > 发送时间: 星期五, 2023年 7 月 21日 下午 3:41:11 > > 主题: [V

Re: [DISCUSS][2.0] FLIP-343: Remove parameter in WindowAssigner#getDefaultTrigger()

2023-07-21 Thread Yuxin Tan
+1 Best, Yuxin Jing Ge 于2023年7月21日周五 15:41写道: > +1 > > NIT: the release in the FLIP is still empty, it should be 2.0 > > Best regards, > Jing > > On Fri, Jul 21, 2023 at 6:03 AM Xintong Song > wrote: > > > +1 > > > > Best, > > > > Xintong > > > > > > > > On Fri, Jul 21, 2023 at 10:53 AM Wenco

Re: [DISCUSS][2.0] FLIP-347: Remove IOReadableWritable serialization in Path

2023-07-21 Thread Yuxin Tan
+1 Best, Yuxin Xintong Song 于2023年7月21日周五 12:04写道: > +1 > > Best, > > Xintong > > > > On Fri, Jul 21, 2023 at 10:54 AM Wencong Liu wrote: > > > Hi devs, > > > > I would like to start a discussion on FLIP-347: Remove IOReadableWritable > > serialization in Path [1]. > > > > > > The Path class

Re: [DISCUSS][2.0] FLIP-344: Remove parameter in RichFunction#open

2023-07-21 Thread Yuxin Tan
+1 Best, Yuxin Xintong Song 于2023年7月21日周五 12:04写道: > +1 > > Best, > > Xintong > > > > On Fri, Jul 21, 2023 at 10:52 AM Wencong Liu wrote: > > > Hi devs, > > > > I would like to start a discussion on FLIP-344: Remove parameter in > > RichFunction#open [1]. > > > > The open() method in RichFunc

Re: [DISCUSS][2.0] FLIP-347: Remove IOReadableWritable serialization in Path

2023-07-21 Thread Matthias Pohl
There's a kind-of-related issue FLINK-4758 [1] that proposes removing the IOReadableWritable interface from more classes. It was briefly mentioned in the must-have work items discussion [2]. I'm not too sure about the usage of IOReadableWritable: ...whether it would go away with the removal of the

Re: [VOTE] Release 2.0 must-have work items

2023-07-21 Thread Alexander Fedulov
> How about this, we continue with the vote as is, and keep the discussion on the SourceFunction in Jira or a separate thread. Sure, but I just want to mention two important things here before we switch over to [1]: >Given that eliminating the removal of SourceFunction was proposed 10 days - This

[DISCUSS] Feature freeze and deprecation work for 2.0

2023-07-21 Thread Matthias Pohl
The feature freeze was postponed to July 24 (end of this week in Europe/early morning Monday in East Asia) in [1]. What's the 1.18 release managers' take on all the FLIPs that were recently started and require some deprecation work (which ideally should go into 1.18)? How does that work with the fe

Re: FLIP-342: Remove brackets around keys returned by MetricGroup#getAllVariables

2023-07-21 Thread Chesnay Schepler
There's a separate discussion altogether. On 21/07/2023 02:49, Mason Chen wrote: Hi all, I agree getScopeVariables is in line with the existing terminology but I’ve seen that the existing terminology is a bit confusing with regards to how users end up querying these metrics and building alerts/

Re: [DISCUSS] Feature freeze and deprecation work for 2.0

2023-07-21 Thread Xintong Song
Good question. CC-ed the release managers. My 2-cents: I think the purpose of feature freeze is to prevent new feature / improvement changes from destabilizing the code base, in order to get a stable and verified release. Based on this, I'd suggest: - Considering FLIPs that purely mark an API as d

Re: [VOTE] Release 2.0 must-have work items

2023-07-21 Thread Leonard Xu
> @Leonard > If we follow the agreed-upon and voted path and do not revert [1], all the > formalities get fulfilled Hi Alexander Sorry for making you uncomfortable even though it was discussed on the dev list and under JIRA before reverting this PR. I'm also +1 for the FLIP as your posted.

Re: [DISCUSS][2.0] FLIP-347: Remove IOReadableWritable serialization in Path

2023-07-21 Thread Jing Ge
Hi Wencong, Just out of curiosity, will the newly introduced deserializeFromDataInputView() method make the Path mutable again? What Matthias suggested makes sense, although the extension might make this FLIP cross multiple modules. Best regards, Jing On Fri, Jul 21, 2023 at 10:23 AM Matthias P

Re: [VOTE] Release 2.0 must-have work items

2023-07-21 Thread Chesnay Schepler
On 21/07/2023 11:45, Leonard Xu wrote: In this way, the user will see the deprecated API firstly but they can not find a candidate if we can not finish all tasks in one minor version . i'm not convinced that this matters. There will be a whole bunch of APIs deprecated in 1.18 (that will remai

Re: [DISCUSS][2.0] FLIP-344: Remove parameter in RichFunction#open

2023-07-21 Thread Jing Ge
+1 On Fri, Jul 21, 2023 at 10:22 AM Yuxin Tan wrote: > +1 > > Best, > Yuxin > > > Xintong Song 于2023年7月21日周五 12:04写道: > > > +1 > > > > Best, > > > > Xintong > > > > > > > > On Fri, Jul 21, 2023 at 10:52 AM Wencong Liu > wrote: > > > > > Hi devs, > > > > > > I would like to start a discussion o

Re: [DISCUSS] Status of Statefun Project

2023-07-21 Thread Filip Karnicki
This is great news, as we're using statefun as well. Please don't hesitate to let me know if you need me to do some additional testing on a real life prod-like setup. On Sat, 24 Jun 2023 at 18:41, Galen Warren via user wrote: > Great -- thanks! > > I'm going to be out of town for about a week b

[jira] [Created] (FLINK-32641) json format supports pojo type

2023-07-21 Thread Jacky Lau (Jira)
Jacky Lau created FLINK-32641: - Summary: json format supports pojo type Key: FLINK-32641 URL: https://issues.apache.org/jira/browse/FLINK-32641 Project: Flink Issue Type: Improvement Co

Re: [DISUCSS] Deprecate multiple APIs in 1.18

2023-07-21 Thread Jing Ge
got it, thanks for the clarification. Best regards, Jing On Fri, Jul 21, 2023 at 5:14 AM Jane Chan wrote: > Hi, Jing, > > I share the same opinion with Xintong. The base class `TimestampExtractor` > for these two classes was deprecated in release-1.12[1][2], so deprecating > these two subclasse

Re:Re: [DISCUSS][2.0] FLIP-347: Remove IOReadableWritable serialization in Path

2023-07-21 Thread Wencong Liu
Hello Matthias, Thanks for your response. As described in FLIP-347, I have thoroughly reviewed all instances where the IOReadableWritable serialization of the Path class is used, and I have identified three classes that still utilize it for de/serializing the Path. 1. FileSourceSplitSerial

Re:Re: [DISCUSS][2.0] FLIP-347: Remove IOReadableWritable serialization in Path

2023-07-21 Thread Wencong Liu
Hello Jing, Thanks for your reply. The URI field should be final and the Path will be immutable.The static method deserializeFromDataInputView will create a new Path object instead of replacing the URI field in a existed Path Object. For the crossing multiple modules issue, I've explained it in

Re: [DISCUSS] FLIP-327: Support stream-batch unified operator to improve job throughput when processing backlog data

2023-07-21 Thread Piotr Nowojski
Hi Dong, Thanks a lot for the answers. I can now only briefly answer your last email. > It is possible that spilling to disks might cause larger overhead. IMO it > is an orthogonal issue already existing in Flink. This is because a Flink > job running batch mode might also be slower than its thro

[jira] [Created] (FLINK-32642) The upsert mode doesn't work for the compound keys

2023-07-21 Thread jasonliangyc (Jira)
jasonliangyc created FLINK-32642: Summary: The upsert mode doesn't work for the compound keys Key: FLINK-32642 URL: https://issues.apache.org/jira/browse/FLINK-32642 Project: Flink Issue Type

Re: Re: [DISCUSS][2.0] FLIP-347: Remove IOReadableWritable serialization in Path

2023-07-21 Thread Jing Ge
Hi Wencong, Thanks for the clarification. I got your point. It makes sense. Wrt IOReadableWritable, the suggestion was to check all classes that implemented it, e.g. BlockInfo, Value, Configuration, etc. Not limited to the Path. Best regards, Jing On Fri, Jul 21, 2023 at 4:31 PM Wencong Liu wr

Re: Rocksdb info

2023-07-21 Thread Taher Koitawala
Thanks guys! This was really helpful On Fri, 21 Jul 2023 at 8:49 AM, tison wrote: > > This is the case. > > Yanfei is correct that the uploading happens during the checkpoint stage. > > Before checkpointing, there is a snapshotting stage that synchronously > takes RocksDB snapshot by forcing all

[jira] [Created] (FLINK-32643) Introduce off-heap shared state cache across stateful operators in TM

2023-07-21 Thread Fang Yong (Jira)
Fang Yong created FLINK-32643: - Summary: Introduce off-heap shared state cache across stateful operators in TM Key: FLINK-32643 URL: https://issues.apache.org/jira/browse/FLINK-32643 Project: Flink

[jira] [Created] (FLINK-32644) ts

2023-07-21 Thread weitianpei (Jira)
weitianpei created FLINK-32644: -- Summary: ts Key: FLINK-32644 URL: https://issues.apache.org/jira/browse/FLINK-32644 Project: Flink Issue Type: Bug Reporter: weitianpei -- This

退订

2023-07-21 Thread 喻凯