Re: [ANNOUNCE] Dian Fu becomes a Flink committer

2020-01-16 Thread Jingsong Li
Congratulations Dian Fu. Well deserved! Best, Jingsong Lee On Thu, Jan 16, 2020 at 6:26 PM jincheng sun wrote: > Congrats Dian Fu and welcome on board! > > Best, > Jincheng > > Shuo Cheng 于2020年1月16日周四 下午6:22写道: > >> Congratulations! Dian Fu >> >> > Xingbo Wei Zhong 于2020年1月16日周四 下午6:13写道: >

Re: [DISCUSS] Improve TableFactory

2020-01-16 Thread Jingsong Li
ot; in TableSourceFactory (would be > > similar for sink factory) to let planner handle which mode (streaming vs > > batch) of source should be instantiated? That way we don't need to always > > let connector developers handling an if-else on isStreamingMode. > > 2.

Re: [DISCUSS] Change default for RocksDB timers: Java Heap => in RocksDB

2020-01-16 Thread Jingsong Li
Hi Stephan, Thanks for starting this discussion. +1 for stores times in RocksDB by default. In the past, when Flink didn't save the times with RocksDb, I had a headache. I always adjusted parameters carefully to ensure that there was no risk of Out of Memory. Just curious, how much impact of heap

Re: [DISCUSS] Releasing Flink 1.9.2

2020-01-22 Thread Jingsong Li
Thanks Hequn. +1 for starting RC of 1.9.2. With the development of our 1.10 bug fix, we have fixed 137 issues for 1.9.2. It's worth releasing 1.9.2, which really solves a lot of urgent bugs for users. [1] https://issues.apache.org/jira/browse/FLINK-15726?jql=project%20%3D%20FLINK%20AND%20fixVersi

Re: [ANNOUNCE] Yu Li became a Flink committer

2020-01-23 Thread Jingsong Li
Congratulations Yu, well deserved! And thanks for your great contribution to the 1.10 release. Best, Jingsong Lee On Thu, Jan 23, 2020 at 6:14 PM Fabian Hueske wrote: > Congrats Yu! > Good to have you on board! > > Cheers, Fabian > > Am Do., 23. Jan. 2020 um 11:13 Uhr schrieb Piotr Nowojski <

Re: [VOTE] Integrate Flink Docker image publication into Flink release process

2020-01-30 Thread Jingsong Li
+1 (non-binding) Official docker support really looks good to me. Best, Jingsong Lee On Fri, Jan 31, 2020 at 12:55 AM aihua li wrote: > +1 (non-binding) > > > 2020年1月30日 下午7:36,Igal Shilman 写道: > > > > +1 (non-binding) > > > > On Thu, Jan 30, 2020 at 12:18 PM Yu Li wrote: > > > >> +1 (non-bi

Re: [VOTE] Release 1.10.0, release candidate #1

2020-01-31 Thread Jingsong Li
Hi, +1 (non-binding) Thanks for driving this, Gary & Yu. - Verified signatures and checksums - Maven build from source skip tests - Start local cluster and web ui is accessible - Submit example of both batch and streaming, run well and log no exception. - Verified pom files point to the 1.1

Re: [VOTE] Release 1.10.0, release candidate #1

2020-01-31 Thread Jingsong Li
Thanks Jincheng, FLINK-15840 [1] should be a blocker, lead to "TableEnvironment.from/scan(string path)" cannot be used for all temporaryTable and catalogTable (not DataStreamTable). Of course, it can be bypassed by "TableEnvironment.sqlQuery("select * from t")", but "from/scan" are very important

Re: [DISCUSS] Improve TableFactory

2020-02-03 Thread Jingsong Li
onfig(); /** * @return Input whether or not it is bounded. */ boolean isBounded(); } } If there is no objection, I will start a vote thread. (if necessary, I can also edit a FLIP). Best, Jingsong Lee On Thu, Jan 16, 2020 at 7:56 PM Jingsong Li wrote: > Thanks Bowen and Timo for involving

Re: [DISCUSS] Update UpsertStreamTableSink and RetractStreamTableSink to new type system

2020-02-03 Thread Jingsong Li
Hi Zhenghua, The *getRecordDataType* looks good to me. But the main problem is how to represent the tuple type in DataType. I understand that it is necessary to use StructuredType, but at present, planner does not support StructuredType, so the other way is to support StructuredType. Best, Jings

Re: [VOTE] FLIP-27 - Refactor Source Interface

2020-02-03 Thread Jingsong Li
+1 (non-binding), thanks for driving. FLIP-27 is the basis of a lot of follow-up work. Best, Jingsong Lee On Tue, Feb 4, 2020 at 10:26 AM Jark Wu wrote: > Thanks for driving this Becket! > > +1 from my side. > > Cheers, > Jark > > On Mon, 3 Feb 2020 at 18:06, Yu Li wrote: > > > +1, thanks for

Re: [VOTE] Release 1.10.0, release candidate #1

2020-02-03 Thread Jingsong Li
Another critical issue is FLINK-15858[1]. It is indeed a regression. But we don''t want to block release. Will try our best to fix it. [1] https://issues.apache.org/jira/browse/FLINK-15858 Best, Jingsong Lee On Tue, Feb 4, 2020 at 9:56 AM Thomas Weise wrote: > I opened a PR for FLINK-15868 > <

Re: [DISCUSS] Improve TableFactory

2020-02-03 Thread Jingsong Li
Hi all, After rethinking and discussion with Kurt, I'd like to remove "isBounded". We can delay this is bounded message to TableSink. With TableSink refactor, we need consider "consumeDataStream" and "consumeBoundedStream". Best, Jingsong Lee On Mon, Feb 3,

Re: [DISCUSS] Improve TableFactory

2020-02-03 Thread Jingsong Li
. */ ReadableConfig getTableConfig(); } } Best, Jingsong Lee On Tue, Feb 4, 2020 at 1:22 PM Jingsong Li wrote: > Hi all, > > After rethinking and discussion with Kurt, I'd like to remove "isBounded". > We can delay this is bounded message to TableSink. > Wi

[VOTE] Improve TableFactory to add Context

2020-02-03 Thread Jingsong Li
Hi all, I would like to start the vote for the improve of TableFactory, which is discussed and reached a consensus in the discussion thread[2]. The vote will be open for at least 72 hours. I'll try to close it unless there is an objection or not enough votes. [1] http://apache-flink-mailing-list

Re: [DISCUSS] have separate Flink distributions with built-in Hive dependencies

2020-02-04 Thread Jingsong Li
Hi all, For your information, we have document the dependencies detailed information [1]. I think it's a lot clearer than before, but it's worse than presto and spark (they avoid or have built-in hive dependency). I thought about Stephan's suggestion: - The hive/lib has 200+ jars, but we only nee

Re: [DISCUSS] Improve TableFactory

2020-02-04 Thread Jingsong Li
uot; config but might give access to the full > Flink config > > Thanks, > Timo > > > On 04.02.20 06:27, Jingsong Li wrote: > > So the interface will be: > > > > public interface TableSourceFactory extends TableFactory { > > .. > > > >

Re: [VOTE] Improve TableFactory to add Context

2020-02-04 Thread Jingsong Li
Hi all, Interface updated. Please re-vote. Best, Jingsong Lee On Tue, Feb 4, 2020 at 1:28 PM Jingsong Li wrote: > Hi all, > > I would like to start the vote for the improve of > TableFactory, which is discussed and > reached a consensus in the discussion thread[2]. > >

Re: [DISCUSS] Improve TableFactory

2020-02-05 Thread Jingsong Li
Hi all, As Jark suggested in VOTE thread. JIRA created: https://issues.apache.org/jira/browse/FLINK-15912 Best, Jingsong Lee On Wed, Feb 5, 2020 at 10:57 AM Jingsong Li wrote: > Hi Timo, > > G ood catch! > > I really love the idea 2, a full Flink config looks very good to

Re: [DISCUSS] Remove registration of TableSource/TableSink in Table Env and ConnectTableDescriptor

2020-02-05 Thread Jingsong Li
HI Kurt, +1 to remove these methods. But one concern is that some of the current TableSource/TableSink may not be ready, such as the JDBCUpsertTableSink, which accepts a JDBCDialect, but through the TableFactory, there is no way to pass in the JDBCDialect at present. But I also believe we have en

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-01 Thread Jingsong Li
> a Row has two modes represented by an internal boolean flag `hasFieldOrder` +1 confusion with Dawid that what's the result when index-based setters and name-based setters are mixed used. And name-based setters look like append instead of set. It reminds me of Avro's `GenericRecord`, We should s

Re: [DISCUSS] Introduce partitioning strategies to Table/SQL

2020-09-02 Thread Jingsong Li
> separate FLIP for the parititioning topic. I'm currently working on an > update to FLIP-107 and would suggest to remove the paritioning topic > there. FLIP-107 will only focus on accessing metadata and expressing > key/value formats. > > What do you think? > > Regards,

Re: [DISCUSS] Releasing Flink 1.11.2

2020-09-08 Thread Jingsong Li
14942 into 1.11.2. FLINK-14942 (this > >>> fixes a > >>> >> bug introduce in 1.11.0), there is a pr for it already. > >>> >> Best, > >>> >> Congxian > >>> >> > >>> >> > >>> >> Zhou, Brian 于2020

Re: [DISCUSS] Releasing Flink 1.11.2

2020-09-08 Thread Jingsong Li
11:08 AM Jingsong Li wrote: > Hi Zhu Zhu, > > Add a new blocker: https://issues.apache.org/jira/browse/FLINK-19166 > > Will fix it soon. > > Best, > Jingsong > > On Tue, Sep 8, 2020 at 12:26 AM Zhu Zhu wrote: > >> Hi All, >> >> Since there a

Re: [ANNOUNCE] New Apache Flink Committer - Arvid Heise

2020-09-15 Thread Jingsong Li
Congratulations Arvid ! Best, Jingsong On Tue, Sep 15, 2020 at 3:27 PM Dawid Wysakowicz wrote: > Congratulations Arvid! Very well deserved! > > Best, > > Dawid > > On 15/09/2020 04:38, Zhijiang wrote: > > Hi all, > > > > On behalf of the PMC, I’m very happy to announce Arvid Heise as a new > Fl

Re: [ANNOUNCE] New Apache Flink Committer - Godfrey He

2020-09-15 Thread Jingsong Li
Congratulations :) Best, Jingsong On Wed, Sep 16, 2020 at 12:45 PM Dian Fu wrote: > Congratulations, well deserved! > > Regards, > Dian > > > 在 2020年9月16日,下午12:36,Guowei Ma 写道: > > > > Congratulations :) > > > > Best, > > Guowei > > > > > > On Wed, Sep 16, 2020 at 12:19 PM Jark Wu wrote: > >

Re: [ANNOUNCE] Apache Flink 1.11.2 released

2020-09-16 Thread Jingsong Li
Thanks ZhuZhu for driving the release. Best, Jingsong On Thu, Sep 17, 2020 at 1:29 PM Zhu Zhu wrote: > The Apache Flink community is very happy to announce the release of Apache > Flink 1.11.2, which is the second bugfix release for the Apache Flink 1.11 > series. > > Apache Flink® is an open-s

Re: [VOTE] FLIP-107: Handling of metadata in SQL connectors

2020-09-17 Thread Jingsong Li
+1 It is very useful Best, Jingsong On Thu, Sep 17, 2020 at 11:12 AM Jark Wu wrote: > +1 (binding) > > Best, > Jark > > On Tue, 15 Sep 2020 at 10:32, Leonard Xu wrote: > > > +1(non-binding) > > > > Leonard > > > > > 在 2020年9月12日,21:46,Danny Chan 写道: > > > > > > +1, non-binding ~ > > > > > > K

[DISCUSS] What is the correct site version?

2020-09-17 Thread Jingsong Li
Hi Dev, Flink 1.11.2 has been released, (thanks ZhuZhu) But I found the site version on https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive/#using-bundled-hive-jar is still 1.11.0. (1.10 is the same) For example: https://repo.maven.apache.org/maven2/org/apache/flink/flink-

Re: [DISCUSS] What is the correct site version?

2020-09-17 Thread Jingsong Li
ld update the "version" in docs/_config.yml in release-1.11 > branch. > > Best, > Jark > > On Fri, 18 Sep 2020 at 14:00, Chesnay Schepler wrote: > >> > Do we need to change this site version after releasing minor/bugfix >> >> versions? >>

[DISCUSS] FLIP-146: Improve new TableSource and TableSink interfaces

2020-09-23 Thread Jingsong Li
Hi all, I'd like to start a discussion about improving the new TableSource and TableSink interfaces. Most connectors have been migrated to FLIP-95, but there are still the Filesystem and Hive that have not been migrated. They have some requirements on table connector API. And users also have some

Re: Re: [DISCUSS] Support source/sink parallelism config in Flink sql

2020-09-23 Thread Jingsong Li
Hi , I have started a discussion about improving the new TableSource and TableSink: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-146-Improve-new-TableSource-and-TableSink-interfaces-td45161.html It includes parallelism setting, welcome to join the discussion and look

[DISCUSS] Move Hive document to "Table & SQL Connectors" from "Table API & SQL"

2020-09-23 Thread Jingsong Li
Hi devs and users: After the 1.11 release, I heard some voices recently: How can't Hive's documents be found in the "Table & SQL Connectors". Actually, Hive's documents are in the "Table API & SQL". Since the "Table & SQL Connectors" document was extracted separately, Hive is a little out of plac

Re: [VOTE] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-24 Thread Jingsong Li
+1 (binding) Best, Jingsong On Thu, Sep 24, 2020 at 4:18 PM Kurt Young wrote: > +1 (binding) > > Best, > Kurt > > > On Thu, Sep 24, 2020 at 4:01 PM Timo Walther wrote: > > > Hi all, > > > > after the discussion in [1], I would like to open a second voting thread > > for FLIP-136 [2] which cove

Re: [DISCUSS] FLIP-146: Improve new TableSource and TableSink interfaces

2020-09-24 Thread Jingsong Li
your > opinion. > > > > > > Regarding the interface `DataStreamScanProvider`, a concrete example > > would > > > help the discussion. What kind > > > of scenarios do you want to support? And what kind of connectors need > > such > > > an int

Re: [DISCUSS] FLIP-146: Improve new TableSource and TableSink interfaces

2020-09-25 Thread Jingsong Li
Hi Aljoscha, Thank you for your feedback, ## Connector parallelism Requirements: Set parallelism by user specified or inferred by connector. How to configure parallelism in DataStream: In the DataStream world, the only way to configure parallelism is `SingleOutputStreamOperator.setParallelism`.

Re: [DISCUSS] Move Hive document to "Table & SQL Connectors" from "Table API & SQL"

2020-09-25 Thread Jingsong Li
t 2:49 AM Rui Li wrote: > > > >> +1 > >> > >> On Thu, Sep 24, 2020 at 2:59 PM Timo Walther > wrote: > >> > >>> +1 > >>> > >>> On 24.09.20 06:54, Jark Wu wrote: > >>>> +1 to move it there. > >>

Re: [DISCUSS][Code-Style] The approach to implement singleton pattern

2020-09-25 Thread Jingsong Li
Hi, thanks for starting this discussion. I am +1 for using the private constructor for util class. We don't need to change it. I think few libraries use the enum, such as guava, common-utils, or even JDK, the private constructor is widely used. I don't quite understand why a util class is an enu

Re: [VOTE] FLIP-143: Unified Sink API

2020-09-28 Thread Jingsong Li
+1 (binding) Best, Jingsong On Mon, Sep 28, 2020 at 3:21 AM Kostas Kloudas wrote: > +1 (binding) > > @Steven Wu I think there will be opportunities to fine tune the API > during the implementation. > > Cheers, > Kostas > > On Sun, Sep 27, 2020 at 7:56 PM Steven Wu wrote: > > > > +1 (non-bindin

Re: [ANNOUNCE] New PMC member: Zhu Zhu

2020-10-09 Thread Jingsong Li
Congratulations, Zhu Zhu! On Fri, Oct 9, 2020 at 3:08 PM Zhijiang wrote: > Congratulations and welcome, Zhu Zhu! > > Best, > Zhijiang > -- > From:Yun Tang > Send Time:2020年10月9日(星期五) 14:20 > To:dev@flink.apache.org > Subject:Re: [

Re: [DISCUSS] FLIP-146: Improve new TableSource and TableSink interfaces

2020-10-09 Thread Jingsong Li
bleSource > >>or ScanRuntimeProvider? > >>2. `scan.infer-parallelism.enabled` doesn't seem very useful to me. > From > >>a user's perspective, parallelism is either set by > `scan.parallelism`, or > >>automatically decided by F

Re: [DISCUSS] FLIP-145: Support SQL windowing table-valued function

2020-10-10 Thread Jingsong Li
+1 for voting. Thanks Jark for driving. +1 for TVF, It has been put forward by theory and supported by calcite. It will greatly enhance the window related operations. My personal feeling is that after TVF, the following operations can be similar to the traditional batch SQL, as long as the window

Re: [VOTE] FLIP-145: Support SQL windowing table-valued function

2020-10-11 Thread Jingsong Li
+1 Best, Jingsong On Sun, Oct 11, 2020 at 8:02 AM hailongwang <18868816...@163.com> wrote: > +1(non-binding) > > > > Best, > Hailong Wang > At 2020-10-10 17:06:07, "Jark Wu" wrote: > >Hi all, > > > >I would like to start the vote for FLIP-145 [1], which is discussed and > >reached consensus in

Re: [DISCUSS] Remove flink-connector-filesystem module.

2020-10-13 Thread Jingsong Li
Hi, I share a concern: Although we now support ORC Writer. It's not easy to support. We need to override something for ORC classes. Note that we are using a newer version of ORC, which is not forward compatible. Therefore, the data written by users using Flink Orc writer may not be readable by o

Re: [DISCUSS] FLIP-146: Improve new TableSource and TableSink interfaces

2020-10-13 Thread Jingsong Li
idual > operations in the DataStream API because it again takes freedom away > from the framework. So if it's really sth that users need we should go > ahead. > > Best, > Aljoscha > > On 09.10.20 13:57, Jingsong Li wrote: > > Hi Aljoscha, > > > > I want to

[VOTE] FLIP-146: Improve new TableSource and TableSink interfaces

2020-10-15 Thread Jingsong Li
Hi all, I would like to start the vote for FLIP-146 [1], which is discussed and reached consensus in the discussion thread [2]. The vote will be open until 20th Oct. (72h, exclude weekends), unless there is an objection or not enough votes. [1] https://cwiki.apache.org/confluence/display/FLINK/FL

Re: [VOTE] FLIP-146: Improve new TableSource and TableSink interfaces

2020-10-18 Thread Jingsong Li
>> > >>> 2020年10月16日 上午10:05,Danny Chan 写道: > >>> > >>> +1, nice job ! > >>> > >>> Best, > >>> Danny Chan > >>> 在 2020年10月15日 +0800 PM8:08,Jingsong Li ,写道: > >>>> Hi all, > >>>> > >&

Re: [DISCUSS] FLIP-149: Introduce the KTable Connector

2020-10-19 Thread Jingsong Li
Thanks Shengkai for your proposal. +1 for this feature. > Future Work: Support bounded KTable source I don't think it should be a future work, I think it is one of the important concepts of this FLIP. We need to understand it now. Intuitively, a ktable in my opinion is a bounded table rather th

Re: [DISCUSS] Release 1.12 Feature Freeze

2020-10-19 Thread Jingsong Li
Hi Robert, Thanks for your detailed explanation. At present, we are preparing or participating in Flink forward, so +1 for appropriate extension of deadline. Best, Jingsong On Mon, Oct 19, 2020 at 5:36 PM Kurt Young wrote: > Can we change the freeze date to October 30th (Friday next week)? It

Re: Re: [VOTE] FLIP-146: Improve new TableSource and TableSink interfaces

2020-10-19 Thread Jingsong Li
is indeed an urgent need. > For example, In my inner flink jobs (1w+), we need to provide the ability > for users to customize parallelism, and the parallelism for Kafka always > is the number of partitions. > > > Look forwart to it ! > Best, > Hailong Wang > > 在 2020

Re: [DISCUSS] Release 1.12 Feature Freeze

2020-10-20 Thread Jingsong Li
a while since it will prevent a lot of > duplicate > > > > > merging efforts. > > > > > > > > > > Regarding the date: I'm fine with the proposed date but I can also > > see > > > > > that extending it to the end of the week co

Re: [DISCUSS] FLIP-149: Introduce the KTable Connector

2020-10-22 Thread Jingsong Li
t;>>>>>>>>> Hi devs, > >>>>>>>>>>> > >>>>>>>>>>> As many people are still confused about the difference option > >>>>>>>>> behaviours > >>>>>>>>&g

Re: [DISCUSS] FLIP-149: Introduce the KTable Connector

2020-10-22 Thread Jingsong Li
impala_upsert.html > [6]: > > https://help.sap.com/viewer/7c78579ce9b14a669c1f3295b0d8ca16/Cloud/en-US/ea8b6773be584203bcd99da76844c5ed.html > [7]: https://phoenix.apache.org/atomic_upsert.html > [8]: > > https://docs.oracle.com/en/database/other-databases/nosql-database/1

Re: [DISCUSS] FLIP-149: Introduce the KTable Connector

2020-10-23 Thread Jingsong Li
; > >> [1] > >> > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-149%3A+Introduce+the+upsert-kafka+Connector > >> > >> Shengkai Fang 于2020年10月23日周五 下午2:55写道: > >> > >>> Hi, all. > >>> It seems we have reached

Re: [DISCUSS] FLIP-149: Introduce the KTable Connector

2020-10-23 Thread Jingsong Li
9 PM Kurt Young wrote: > To be precise, it means the Kakfa topic should set the configuration > "cleanup.policy" to "compact" not "delete". > > Best, > Kurt > > > On Fri, Oct 23, 2020 at 4:01 PM Jingsong Li > wrote: > > > I

Re: [VOTE]FLIP-149: Introduce the upsert-kafka connector

2020-10-23 Thread Jingsong Li
+1 On Fri, Oct 23, 2020 at 3:52 PM Konstantin Knauf wrote: > +1 > > On Fri, Oct 23, 2020 at 9:36 AM Jark Wu wrote: > > > +1 > > > > On Fri, 23 Oct 2020 at 15:25, Shengkai Fang wrote: > > > > > Hi, all, > > > > > > I would like to start the vote for FLIP-149[1], which is discussed and > > > rea

Re: [VOTE] FLIP-148: Introduce Sort-Merge Based Blocking Shuffle to Flink

2020-10-25 Thread Jingsong Li
+1 Best, Jingsong On Mon, Oct 26, 2020 at 11:52 AM Zhijiang wrote: > Thanks for driving this improvement, Yingjie! > > +1 (binding) > > Best, > Zhijiang > > > -- > From:Kurt Young > Send Time:2020年10月26日(星期一) 11:41 > To:dev > Sub

Re: [DISCUSS] Releasing Apache Flink 1.11.3

2020-10-28 Thread Jingsong Li
+1 to backport the FLIP-27 adjustments to 1.11.x. If possible, that would be great. Many people are looking forward to the FLIP-27 interface, but they don't want to take the risk to upgrade to 1.12 (And wait 1.12). After all, 1.11 is a relatively stable version. Best, Jingsong On Thu, Oct 29, 20

Re: [DISCUSS] Remove flink-connector-filesystem module.

2020-10-28 Thread Jingsong Li
+1 to remove the Bucketing Sink. Thanks for the effort on ORC and `HadoopPathBasedBulkFormatBuilder`, I think it's safe to get rid of the old Bucketing API with them. Best, Jingsong On Thu, Oct 29, 2020 at 3:06 AM Kostas Kloudas wrote: > Thanks for the discussion! > > From this thread I do not

Re: [ANNOUNCE] New Apache Flink Committer - Congxian Qiu

2020-10-29 Thread Jingsong Li
Congratulations! On Thu, Oct 29, 2020 at 3:56 PM Yuan Mei wrote: > Congratulations! > > On Thu, Oct 29, 2020 at 3:53 PM Till Rohrmann > wrote: > > > Congratulations Congxian! Great to have you as a committer now :-) > > > > Cheers, > > Till > > > > On Thu, Oct 29, 2020 at 8:33 AM Benchao Li wr

Re: [VOTE] Remove flink-connector-filesystem module.

2020-10-29 Thread Jingsong Li
+1 On Thu, Oct 29, 2020 at 7:59 PM Chesnay Schepler wrote: > +1 > > On 10/29/2020 9:18 AM, Kostas Kloudas wrote: > > Hi all, > > > > Following the discussion in [1], I would like to start a vote on > > removing the flink-connector-filesystem module which includes the > > BucketingSink. > > > > T

Re: Re: Re: [VOTE] FLIP-145: Support SQL windowing table-valued function (2nd)

2020-11-11 Thread Jingsong Li
+1 On Thu, Nov 12, 2020 at 2:07 PM hailongwang <18868816...@163.com> wrote: > > > +1(no-binding) > > > > > 在 2020-11-11 19:44:40,"刘大龙" 写道: > > > >+1 > > > >> -原始邮件- > >> 发件人: "Timo Walther" > >> 发送时间: 2020-11-11 18:55:06 (星期三) > >> 收件人: dev@flink.apache.org > >> 抄送: > >> 主题: Re: [VOTE]

Re: Hive Streaming write compaction

2020-11-18 Thread Jingsong Li
Hi Chen, Table Filesystem/Hive sink file compaction has been merged into master, detail in [1]. It is included in Flink 1.12. Hope you can have a try and test. [1]https://issues.apache.org/jira/browse/FLINK-19345 Best, Jingsong On Thu, Nov 19, 2020 at 2:31 PM Chen Qin wrote: > Hi there, > >

Re: [DISCUSS] Releasing Apache Flink 1.12.1

2020-12-17 Thread Jingsong Li
Thanks for volunteering as our release manager Xintong. +1 for releasing Flink 1.12.1 soon. I think https://issues.apache.org/jira/browse/FLINK-20665 should be addressed, I marked it as a Blocker. Best, Jingsong On Fri, Dec 18, 2020 at 11:16 AM Yang Wang wrote: > Hi David, > > I will take a lo

Re: [DISCUSS] Allow streaming operators to use managed memory

2021-01-04 Thread Jingsong Li
+1 for allowing streaming operators to use managed memory. The memory use of streams requires some hierarchy, and the bottom layer is undoubtedly the current StateBackend. Let the stream operators freely use the managed memory, which will make the memory management model to be unified and give the

Re: Support local aggregate push down for Blink batch planner

2021-01-04 Thread Jingsong Li
Thanks for your proposal! Sebastian. +1 for SupportsAggregatePushDown. The above wonderful discussion has solved many of my concerns. ## Semantic problems We may need to add some mechanisms or comments, because as far as I know, the semantics of each database is actually different, which may nee

Re: Support local aggregate push down for Blink batch planner

2021-01-05 Thread Jingsong Li
nterface, > and holds the > `List < CallExpression > aggregateExpressions, int[] GroupingFields, > DataType producedDataType` > fields? > > Looking forward to your further feedback or guidance. > > Jingsong Li 于2021年1月5日周二 下午2:44写道: > >> Thanks for your proposal! Seb

Re: Support local aggregate push down for Blink batch planner

2021-01-05 Thread Jingsong Li
as a cost of losing user convenience. > Foremost, we don't see any parameters to add in the future. Do you know > any potential parameters? > > Best, > Jark > > On Wed, 6 Jan 2021 at 10:28, Jingsong Li wrote: > >> Hi Sebastian, >> >> Well, I mean: >> &

Re: Support local aggregate push down for Blink batch planner

2021-01-05 Thread Jingsong Li
Hi, I'm also curious about aggregate with filter (COUNT(1) FILTER(WHERE d > 1)). Can we push it down? I'm not sure that a single call expression can express it, and how we should embody it and convey it to users. Best, Jingsong On Wed, Jan 6, 2021 at 1:36 PM Jingsong Li wrot

Re: Support local aggregate push down for Blink batch planner

2021-01-05 Thread Jingsong Li
t the existing > connector will produce wrong results > when upgrading to new Flink versions (as we are pushing > grouping_sets/filter_args, but connector ignores it). > I think for these cases, providing a new default method to override might > be a better choice. > > Best, >

Re: Support local aggregate push down for Blink batch planner

2021-01-06 Thread Jingsong Li
at present. >>> >>> For core points (b & c): I think we can change the interface to be: >>> ``` >>> >>> boolean applyAggregates(int[] groupingFields, List >>> aggregateExpressions, DataType producedDataType, List >>> groupingSets);

Re: [ANNOUNCE] Welcome Danny Cranmer as a new Apache Flink Committer

2021-01-12 Thread Jingsong Li
Congrats, Danny! Best, Jingsong On Tue, Jan 12, 2021 at 7:55 PM Leonard Xu wrote: > Congratulations Danny! > > Best, > Leonard > > 在 2021年1月12日,19:44,Wei Zhong 写道: > > > > Congratulations Danny! > > > > Best, > > Wei > > > >> 在 2021年1月12日,19:09,hailongwang <18868816...@163.com> 写道: > >> > >> C

Re: [VOTE] FLIP-273: Improve Catalog API to Support ALTER TABLE syntax

2022-11-30 Thread Jingsong Li
+1 (binding) Thanks Shengkai for driving this FLIP. Best, Jingsong On Thu, Dec 1, 2022 at 12:20 PM Shengkai Fang wrote: > > Hi All, > > Thanks for all the feedback so far. Based on the discussion[1] we seem > to have a consensus, so I would like to start a vote on FLIP-273. > > The vote will la

Re: [DISCUSS] the bug with column lost comment

2022-12-07 Thread Jingsong Li
Hi, https://github.com/apache/flink/pull/21182 is being reviewed. You can take a look. Best, Jingsong On Wed, Dec 7, 2022 at 4:46 PM 黑猪 wrote: > > [DISCUSS] > i want to fix the bug with column lost comment,And I have some ideas, use > the new ResolvedSchema to be compatible with existing usa

Re: [DISCUSSION] Scan mode in Table Store for Flink Stream and Batch job

2022-12-09 Thread Jingsong Li
s > a) scan.timestamp-millis, log.scan.timestamp-millis. Optional timestamp > used in case of \"from-timestamp\" scan mode. > b) read.compacted. Read the latest compact snapshots only. > > After discussing with @Jingsong Li and @Caizhi wen, we found that the config &g

Re: [DISCUSS] FLIP-282: Introduce Delete & Update API

2023-01-02 Thread Jingsong Li
Thanks yuxia for the FLIP! It looks really good! I have three comments: ## RowLevelDeleteMode Can RowLevelDeleteMode be a higher level? `SupportsRowLevelDelete.RowLevelDeleteMode` is better than `SupportsRowLevelDelete.RowLevelDeleteInfo.RowLevelDeleteMode`. Same as `RowLevelUpdateMode`. ## Sco

Re: [DISCUSS] Extending the feature freezing date of Flink 1.17

2023-01-03 Thread Jingsong Li
+1 On Tue, Jan 3, 2023 at 4:40 PM Matthias Pohl wrote: > > +1 for extending to Jan 31 > > On Tue, Jan 3, 2023 at 8:33 AM Yu Li wrote: > > > +1 for the proposal (extending the 1.17 feature freeze date to Jan 31st). > > > > Best Regards, > > Yu > > > > > > On Tue, 3 Jan 2023 at 15:11, Zhu Zhu wro

Re: [DISCUSS] FLIP-280: Introduce a new explain mode to provide SQL advice

2023-01-03 Thread Jingsong Li
Thanks Jane for the FLIP! It looks very nice! Can you give examples of other systems for the syntax? In other systems, is EXPLAIN ANALYZE already PHYSICAL_PLAN? `EXPLAIN ANALYZED_PHYSICAL_PLAN ` looks a bit strange, and even stranger that it contains `advice`. The purpose of FLIP seems to be a b

Re: [DISCUSS] FLIP-282: Introduce Delete & Update API

2023-01-05 Thread Jingsong Li
t; what's scan purpose without touching any other logic. If sources ignore this > method, it just works as normally. So I think there's > no necessary to add a new interface like SupportsXX. > > Best regards, > Yuxia > > - 原始邮件 - > 发件人: "Jingsong Li&quo

Re: [DISCUSS] FLIP-280: Introduce a new explain mode to provide SQL advice

2023-01-06 Thread Jingsong Li
internal > interface, which is different from implementing a custom connector/format. > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-280%3A+Introduce+EXPLAIN+PLAN_ADVICE+to+provide+SQL+advice > [2] https://dev.mysql.com/doc/refman/8.0/en/explain.html#explain-analyze &

Re: [VOTE] FLIP-280: Introduce EXPLAIN PLAN_ADVICE to provide SQL advice

2023-01-09 Thread Jingsong Li
+1 (binding) On Mon, Jan 9, 2023 at 6:19 PM Jane Chan wrote: > > Hi all, > > Thanks for all the feedback so far. > Based on the discussion[1], we have come to a consensus, so I would like to > start a vote on FLIP-280: Introduce EXPLAIN PLAN_ADVICE to provide SQL > advice[2]. > > The vote will la

Re: [VOTE] FLIP-282: Introduce Delete & Update API

2023-01-09 Thread Jingsong Li
+1 binding On Mon, Jan 9, 2023 at 6:14 PM Samrat Deb wrote: > > +1 (non binding ) > > thank you for driving > > > > On Mon, 9 Jan 2023 at 3:36 PM, yuxia wrote: > > > Hi, all. > > > > I'd like to start a vote on FLIP-282: Introduce Delete & Update API[1]. > > You can find the discussion on it in

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

2023-01-09 Thread Jingsong Li
Congratulations, Lincoln! Best, Jingsong On Tue, Jan 10, 2023 at 11:56 AM Leonard Xu wrote: > > Congratulations, Lincoln! > > Impressive work in streaming semantics, well deserved! > > > Best, > Leonard > > > > On Jan 10, 2023, at 11:52 AM, Jark Wu wrote: > > > > Hi everyone, > > > > On behalf

[VOTE] Apache Flink Table Store 0.3.0, release candidate #1

2023-01-10 Thread Jingsong Li
Hi everyone, Please review and vote on the release candidate #1 for the version 0.3.0 of Apache Flink Table Store, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) **Release Overview** As an overview, the release consists of the follo

Re: [VOTE] Apache Flink Table Store 0.3.0, release candidate #1

2023-01-11 Thread Jingsong Li
d, could you double > check and republish if necessary? Thanks. > > Best Regards, > Yu > > [1] https://repository.apache.org/content/repositories/orgapacheflink-1576/ > > > On Tue, 10 Jan 2023 at 16:53, Jingsong Li wrote: > > > Hi everyone, > > > >

Re: [VOTE] Apache Flink Table Store 0.3.0, release candidate #1

2023-01-12 Thread Jingsong Li
> - Checked the website updates: *OK* > > Best Regards, > Yu > > > On Thu, 12 Jan 2023 at 15:36, Jingsong Li wrote: > > > Thanks Yu for your validation. > > > > I created a new staging directory [1] > > > > [1] > > https://repository.ap

Re: [VOTE] Apache Flink Table Store 0.3.0, release candidate #1

2023-01-13 Thread Jingsong Li
bsite release PR: *OK* > - Checked the licenses: *OK* > - Went through the quick start: *OK* > * Verified with both flink 1.14.5 and 1.15.1 using > * Verified web UI and log output, nothing unexpected > > Best, > Jark > > On Thu, 12 Jan 2023 at 20:52, Jingsong Li wrote: >

[RESULT][VOTE] Apache Flink Table Store 0.3.0, release candidate #1

2023-01-13 Thread Jingsong Li
I'm happy to announce that we have unanimously approved this release. There are 3 approving votes, 3 of which are binding: * Yu Li (binding) * Jark Wu (binding) * Jingsong Lee (binding) There are no disapproving votes. Thank you for verifying the release candidate. I will now proceed to finalize

[ANNOUNCE] Apache Flink Table Store 0.3.0 released

2023-01-13 Thread Jingsong Li
The Apache Flink community is very happy to announce the release of Apache Flink Table Store 0.3.0. Apache Flink Table Store is a unified storage to build dynamic tables for both streaming and batch processing in Flink, supporting high-speed data ingestion and timely data query. Please check out

Re: [ANNOUNCE] New Apache Flink Committer - Jing Ge

2023-02-13 Thread Jingsong Li
Congratulations Jing! Best, Jingsong On Tue, Feb 14, 2023 at 3:50 PM godfrey he wrote: > > Hi everyone, > > On behalf of the PMC, I'm very happy to announce Jing Ge as a new Flink > committer. > > Jing has been consistently contributing to the project for over 1 year. > He authored more than 50

Re: [DISCUSS] FLIP-296: Watermark options for table API & SQL

2023-02-22 Thread Jingsong Li
Thanks for your proposal. +1 to yuxia, consider watermark-related hints as option hints. Personally, I am cautious about adding SQL syntax, WATERMARK_PARAMS is also SQL syntax to some extent. We can use OPTIONS to meet this requirement if possible. Best, Jingsong On Thu, Feb 23, 2023 at 10:41

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-22 Thread Jingsong Li
Thanks for the proposal. +1 for the proposal. I am confused about "Proposed TableEnvironment SQL API Changes", can we just use catalog api for this requirement? Best, Jingsong On Thu, Feb 23, 2023 at 10:48 AM Jacky Lau wrote: > > Hi Ran: > Thanks for driving the FLIP. the google doc looks real

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-22 Thread Jingsong Li
atabases() { > return catalogManager > .getCatalog(catalogManager.getCurrentCatalog()) > .get() > .listDatabases() > .toArray(new String[0]); > } > > Best Regards, > Ran Tao > > > Jingsong Li 于

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-23 Thread Jingsong Li
calls. WDYT? > > Best regards, > Jing > > [1] > https://hackernoon.com/object-oriented-tricks-2-law-of-demeter-4ecc9becad85 > > On Thu, Feb 23, 2023 at 6:26 AM Ran Tao wrote: > > > Hi Jingsong. thanks. i got it. > > In this way, there is no need to introduce new

Re: [ANNOUNCE] Flink project website is now powered by Hugo

2023-02-23 Thread Jingsong Li
Thanks Martijn! On Fri, Feb 24, 2023 at 9:46 AM yuxia wrote: > > Thanks Martijn for your work. > > Best regards, > Yuxia > > - 原始邮件 - > 发件人: "Jing Ge" > 收件人: "dev" > 发送时间: 星期五, 2023年 2 月 24日 上午 5:20:30 > 主题: Re: [ANNOUNCE] Flink project website is now powered by Hugo > > Congrats Martij

Re: [Discuss] Some questions on flink-table-store micro benchmark

2023-02-26 Thread Jingsong Li
Thanks Yun. Another way is we can get rid of JMH, something like Spark `org.apache.spark.benchmark.Benchmark` can replace JMH. Best, Jingsong On Mon, Feb 27, 2023 at 1:24 AM Yun Tang wrote: > > Hi dev, > > I just noticed that flink-table-store had introduced the micro benchmark > module [1] to

Re: [Discuss] Some questions on flink-table-store micro benchmark

2023-02-27 Thread Jingsong Li
cond point, how can we track > the performance changes of the micro benchmark? What do you think? > > Best, > Shammon > > On Mon, Feb 27, 2023 at 10:57 AM Jingsong Li wrote: > > > Thanks Yun. > > > > Another way is we can get rid of JMH, something like Spark

Re: [DISCUSS] Deprecate deserialize method in DeserializationSchema

2023-02-28 Thread Jingsong Li
- `T deserialize(byte[] message)` is widely used and it is a public api. It is very friendly for single record deserializers. - `void deserialize(byte[] message, Collector out)` supports multiple records. I think we can just keep them as they are. Best, Jingsong On Tue, Feb 28, 2023 at 3:08 PM

Re: [DISCUSS] FLIP-293: Introduce Flink Jdbc Driver For Sql Gateway

2023-03-01 Thread Jingsong Li
Thanks Shammon for driving. Big +1 for this. I heard that many users want to use FlinkGateway + JDBC to do some queries, but at present, only Hive JDBC can be used. It is Hive dialect by default, and the experience is also different from FlinkSQL. We need to have our own JDBC. I took a look at y

Re: [VOTE] Flink minor version support policy for old releases

2023-03-02 Thread Jingsong Li
+1 (binding) On Thu, Mar 2, 2023 at 1:30 PM Yu Li wrote: > > +1 (binding) > > Best Regards, > Yu > > > On Thu, 2 Mar 2023 at 09:53, Jark Wu wrote: > > > +1 (binding) > > > > Best, > > Jark > > > > > 2023年3月2日 05:03,Gyula Fóra 写道: > > > > > > +1 (binding) > > > > > > Gyula > > > > > > On Wed, Ma

<    1   2   3   4   5   6   7   >