Re: Merge SupportsComputedColumnPushDown and SupportsWatermarkPushDown

2020-09-04 Thread Shengkai Fang
Hi, all. It seems the format is not normal. So I add a google doc in link[1]. This discussion is about the interfaces in FLIP-95: New Table Source And Table Sink and propose to merge two interfaces SupportsWatermarkPushDown and SupportsComputedColumnPushDown. I am looking forward to any opinions

Re: [DISCUSS] FLIP-140: Introduce bounded style execution for keyed streams

2020-09-04 Thread Aljoscha Krettek
Seth is right, I was just about to write that as well. There is a problem, though, because some of our TypeSerializers are not deterministic even though we use them as if they were. Beam excludes the FloatCoder, for example, and the AvroCoder in certain cases. I'm pretty sure there is also weir

Re: [DISCUSS] FLIP-33: Standardize connector metrics

2020-09-04 Thread Becket Qin
Hi all, To complete the Source refactoring work, I'd like to revive this discussion. Since the mail thread has been dormant for more than a year, just to recap the motivation of the FLIP: 1. The FLIP proposes to standardize the connector metrics by giving guidance on the metric specifications, in

Re: Apply for contributor permissions

2020-09-04 Thread jun su
Hi timo, Got it, thanks for your reply 在 2020年9月4日星期五,Timo Walther 写道: > Hi Jun, > > you don't need contributor permissions to contribute to Apache Flink. > > Just open/pick an issue and ping a committer to assign you to this ticket. > > Regards, > Timo > > On 04.09.20 14:28, jun su wrote

Re: [DISCUSS] FLIP-140: Introduce bounded style execution for keyed streams

2020-09-04 Thread Seth Wiesman
There is already an implicit assumption the TypeSerializer for keys is stable/deterministic, RocksDB compares keys using their serialized byte strings. I think this is a non-issue (or at least it's not changing the status quo). On Fri, Sep 4, 2020 at 6:39 AM Timo Walther wrote: > +1 for getting

Re: Apply for contributor permissions

2020-09-04 Thread Timo Walther
Hi Jun, you don't need contributor permissions to contribute to Apache Flink. Just open/pick an issue and ping a committer to assign you to this ticket. Regards, Timo On 04.09.20 14:28, jun su wrote: Hi Guys, I want to contribute to Apache Flink. Would you please give me the permission as a

Apply for contributor permissions

2020-09-04 Thread jun su
Hi Guys, I want to contribute to Apache Flink. Would you please give me the permission as a contributor? My JIRA ID is sujun1020. -- Best, Jun Su

[jira] [Created] (FLINK-19142) Investigate slot hijacking from preceding pipelined regions after failover

2020-09-04 Thread Andrey Zagrebin (Jira)
Andrey Zagrebin created FLINK-19142: --- Summary: Investigate slot hijacking from preceding pipelined regions after failover Key: FLINK-19142 URL: https://issues.apache.org/jira/browse/FLINK-19142 Proj

Re: [DISCUSS] FLIP-140: Introduce bounded style execution for keyed streams

2020-09-04 Thread Timo Walther
+1 for getting rid of the TypeComparator interface and rely on the serialized representation for grouping. Adding a new type to DataStream API is quite difficult at the moment due to too many components that are required: TypeInformation (tries to deal with logical fields for TypeComparators),

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-04 Thread Aljoscha Krettek
I like the proposal! I didn't check the implementation section in detail but the SQL DDL examples look good as well as the options for specifying how fields are mapped to keys/values look good. Aljoscha On 04.09.20 11:47, Dawid Wysakowicz wrote: Hi Timo, Thank you very much for the update. I

Re: [DISCUSS] FLIP-36 - Support Interactive Programming in Flink Table API

2020-09-04 Thread Timo Walther
Hi Xuannan, thanks for the update. Here is some final feedback from my side. Maybe others have some final feedback as well before we continue to a voting? Some mistakes that we should fix in the FLIP: - The FLIP declares `Table cache();` but I guess this should be `CachedTable cache();` now.

[jira] [Created] (FLINK-19141) Flink Job Submitted on Yarn Does not Work

2020-09-04 Thread Yunhui (Jira)
Yunhui created FLINK-19141: -- Summary: Flink Job Submitted on Yarn Does not Work Key: FLINK-19141 URL: https://issues.apache.org/jira/browse/FLINK-19141 Project: Flink Issue Type: Bug Compo

Re: [DISCUSS] FLIP-140: Introduce bounded style execution for keyed streams

2020-09-04 Thread Aljoscha Krettek
Thanks for publishing the FLIP! On 2020/09/01 06:49:06, Dawid Wysakowicz wrote: > 1. How to sort/group keys? What representation of the key should we > use? Should we sort on the binary form or should we depend on > Comparators being available. Initially, I suggested to Dawid (in priva

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-04 Thread Dawid Wysakowicz
Hi Timo, Thank you very much for the update. It indeed covers the full story in more details. I agree with the proposal. On 04/09/2020 10:48, Timo Walther wrote: > Hi everyone, > > I completely reworked FLIP-107. It now covers the full story how to > read and write metadata from/to connectors and

[jira] [Created] (FLINK-19140) Join with Table Function (UDTF) SQL example is incorrect

2020-09-04 Thread Benchao Li (Jira)
Benchao Li created FLINK-19140: -- Summary: Join with Table Function (UDTF) SQL example is incorrect Key: FLINK-19140 URL: https://issues.apache.org/jira/browse/FLINK-19140 Project: Flink Issue Ty

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-04 Thread Timo Walther
Hi everyone, I completely reworked FLIP-107. It now covers the full story how to read and write metadata from/to connectors and formats. It considers all of the latest FLIPs, namely FLIP-95, FLIP-132 and FLIP-122. It introduces the concept of PERSISTED computed columns and leaves out partition

[jira] [Created] (FLINK-19139) flink-conf.yml read values from environment variables

2020-09-04 Thread Mohammad Hossein Gerami (Jira)
Mohammad Hossein Gerami created FLINK-19139: --- Summary: flink-conf.yml read values from environment variables Key: FLINK-19139 URL: https://issues.apache.org/jira/browse/FLINK-19139 Project: F

Re: [DISCUSS] Releasing Flink 1.11.2

2020-09-04 Thread Zhu Zhu
@Dawid Thanks for the information and preparing a fix for FLINK-19133! I have made it a blocker for 1.11.2 and will keep tracking its status. @Till Thanks for the updates and efforts for FLINK-18959! Thanks, Zhu Till Rohrmann 于2020年9月4日周五 下午3:41写道: > Fyi, FLINK-18959 has been merged into the r

Re: [DISCUSS] Releasing Flink 1.11.2

2020-09-04 Thread Till Rohrmann
Fyi, FLINK-18959 has been merged into the release-1.10 branch. Cheers, Till On Thu, Sep 3, 2020 at 2:38 PM Dawid Wysakowicz wrote: > User has just reported another issue FLINK-19133 which I think should be > a blocker for the 1.11.2 release. I'll try to prepare a fix as soon as > possible. > >

Re: [VOTE] FLIP-138: Declarative Resource Management

2020-09-04 Thread Zhu Zhu
+1 Thanks, Zhu Xintong Song 于2020年9月4日周五 下午1:26写道: > Thanks for starting this vote. > > +1 from my side. > > Thank you~ > > Xintong Song > > > > On Fri, Sep 4, 2020 at 12:37 AM Till Rohrmann > wrote: > > > Hi devs, > > > > I'd like to start a voting thread on FLIP-138 [1], which proposes to ma