Re: [REMINDER] Build docs before merging

2020-11-24 Thread Jark Wu
Is it possible to move it to PR CI? I have seen many documentation link issues these days. Best, Jark On Tue, 24 Nov 2020 at 15:56, Robert Metzger wrote: > Actually, the check_links.sh script is called in the nightly CI runs > > > https://github.com/apache/flink/blob/master

Re: [REMINDER] Build docs before merging

2020-11-24 Thread Jark Wu
Yes. I have opened a pull request: https://github.com/apache/flink/pull/14204 Best, Jark On Tue, 24 Nov 2020 at 17:38, Robert Metzger wrote: > Yes, that should be very easy to do. Just move the docs_404_check to the > "ci" stage in "build-apache-repo.yml". Maybe it

Re: I create a hotfix pull request in `MiniCluster.java`, thanks!

2020-11-25 Thread Jark Wu
ed to committers in time. Best, Jark On Wed, 25 Nov 2020 at 16:45, 左元 wrote: > The Pull Request Number is #14211. > > > Fix typo `dispatcherResourceManagreComponentRpcServiceFactory` -> > `dispatcherResourceManagerComponentRpcServiceFactor

Re: [DISCUSS] Stop adding new bash-based e2e tests to Flink

2020-11-27 Thread Jark Wu
Thanks Matthias, I have created sub-tasks for the Table SQL related bash-based e2e tests. Best, Jark On Fri, 27 Nov 2020 at 21:25, Matthias Pohl wrote: > Thanks Robert for pushing this. +1 for creating java-based e2e tests. > > In the engine team, we decided to work towards th

Re: [DISCUSS] Make SQL docs Blink only

2020-12-08 Thread Jark Wu
+1 This is a very good idea. Best, Jark On Wed, 9 Dec 2020 at 10:43, Xingbo Huang wrote: > +1 > > This is a very good proposal.In release-1.12, many newly added features are > only supported on the blink planner. For example, the newly added features > of PyFlnk in FLIP-137[1]

[DISCUSS] Allow streaming operators to use managed memory

2020-12-22 Thread Jark Wu
to buffer more data without worrying about OOM and improve the performance a lot. What do you think about allowing streaming operators to use managed memory and exposing it in configuration. Best, Jark [1]: https://ci.apache.org/projects/flink/flink-docs-master/deployment/memory/mem_setup_tm.html#managed-memory

Re: [ANNOUNCE] New formatting rules are now in effect

2020-12-28 Thread Jark Wu
Thanks Aljoscha and Chesnay for the great work! Best, Jark On Tue, 29 Dec 2020 at 11:11, Xintong Song wrote: > Great! Thanks Aljoscha and Chesnay for driving this. > > Thank you~ > > Xintong Song > > > > On Tue, Dec 29, 2020 at 1:28 AM Chesnay Schepler > wrote: &

Re: Support local aggregate push down for Blink batch planner

2020-12-28 Thread Jark Wu
type of the arguments. 3) It would be better to list which connectors will be supported in the plan? Best, Jark On Tue, 29 Dec 2020 at 00:49, Sebastian Liu wrote: > Hi all, > > I'd like to discuss a new feature for the Blink Planner. > Aggregate operator of Flink SQL is c

Re: [ANNOUNCE] New formatting rules are now in effect

2020-12-30 Thread Jark Wu
, classLoader; 4 spaces: wrapClassLoader( () -> environment .getModules() .forEach( (name, entry) -> modules.put(name, createModule(entry.asMap(), classLoader; Best, Jark [1]: https://g

Re: Did Flink 1.11 break backwards compatibility for the table environment?

2020-12-30 Thread Jark Wu
le for SQL queries as the physical plan may change when we introduce even a minor optimization, and we may also change the state structure to have better performance for operators. Best, Jark [1]: https://ci.apache.org/projects/flink/flink-docs-release-1.11/release-notes/flink-1.11.html#added-a-

Re: Did Flink 1.11 break backwards compatibility for the table environment?

2020-12-30 Thread Jark Wu
Hi Guenther, I think it's safe to use legacy mode in your case, because the RowKind is never used in the old planner. Hi Till, It seems that the cross-major-version state incompatibility is not documented. I created FLINK-20823 to update the documentation. Best, Jark On Thu, 31 Dec 20

Re: [DISCUSS] Allow streaming operators to use managed memory

2021-01-04 Thread Jark Wu
Hi Aljoscha, I think we may need to divide `DATAPROC` into `OPERATOR` and `STATE_BACKEND`, because they have different scope (slot vs. operator). But @Xintong Song may have more insights on it. Best, Jark On Mon, 4 Jan 2021 at 20:44, Aljoscha Krettek wrote: > I agree, we should al

Re: [DISCUSS] FLIP-155: Introduce a few convenient operations in Table API

2021-01-04 Thread Jark Wu
vent-time/processing-time temporal join, SQL Hints, window TVF (FLIP-145 [1])? Best, Jark [1]: https://cwiki.apache.org/confluence/display/FLINK/FLIP-145%3A+Support+SQL+windowing+table-valued+function On Mon, 4 Jan 2021 at 22:59, Timo Walther wrote: > Hi Dian, > > thanks for the pro

Re: Support local aggregate push down for Blink batch planner

2021-01-04 Thread Jark Wu
agg function can be obtained from the `CallExpression`. 3) What do you think about renaming `groupSet` to `grouping` or `groupedFields` ? The `groupSet` may confuse users that it relates to "grouping sets". What do you think? Best, Jark On Tue, 5 Jan 2021 at 11:04, Kurt Young

Re: [DISCUSS] Allow streaming operators to use managed memory

2021-01-04 Thread Jark Wu
+1 to Xingtong's proposal! Best, Jark On Tue, 5 Jan 2021 at 12:13, Xintong Song wrote: > +1 for allowing streaming operators to use managed memory. > > As for the consumer names, I'm afraid using `DATAPROC` for both streaming > ops and state backends will not work. Cu

Re: [DISCUSS]Some thoughts about CatalogPartitionSpec

2021-01-04 Thread Jark Wu
Hi Jun, I'm curious why it doesn't work when represented in string? You can get the field type from the CatalogTable#getSchema(), then parse/cast the partition value to the type you want. Best, Jark On Tue, 5 Jan 2021 at 13:43, Jun Zhang wrote: > Hello dev: > Now I enc

Re: [DISCUSS]Some thoughts about CatalogPartitionSpec

2021-01-04 Thread Jark Wu
blems. Re: "the system will throw an exception that the type does not match", could your system store partition values as string type? Best, Jark On Tue, 5 Jan 2021 at 14:09, Jun Zhang wrote: > hi ,Jack: > > If the partition type is int and we pass in a string type, th

Re: Support local aggregate push down for Blink batch planner

2021-01-05 Thread Jark Wu
Thanks for the update. The proposal looks good to me now. Best, Jark On Tue, 5 Jan 2021 at 14:44, Jingsong Li wrote: > Thanks for your proposal! Sebastian. > > +1 for SupportsAggregatePushDown. The above wonderful discussion has > solved many of my concerns. > > ## Semant

Re: Support local aggregate push down for Blink batch planner

2021-01-05 Thread Jark Wu
Best, Jark On Wed, 6 Jan 2021 at 10:28, Jingsong Li wrote: > Hi Sebastian, > > Well, I mean: > > `boolean applyAggregates(int[] groupingFields, List > aggregateExpressions, DataType producedDataType);` > VS > ``` > boolean applyAggregates(Aggregation agg); >

Re: [DISCUSS] Allow streaming operators to use managed memory

2021-01-05 Thread Jark Wu
will be OPERATOR:70,STATE_BACKEND:70,PYTHON:30. The weight for OPERATOR and STATE_BACKEND will be the same value to align with previous behaviors. Best, Jark [1]: https://issues.apache.org/jira/browse/FLINK-20860 On Tue, 5 Jan 2021 at 18:35, Xintong Song wrote: > > > > Would the de

Re: Support local aggregate push down for Blink batch planner

2021-01-05 Thread Jark Wu
upgrading to new Flink versions (as we are pushing grouping_sets/filter_args, but connector ignores it). I think for these cases, providing a new default method to override might be a better choice. Best, Jark On Wed, 6 Jan 2021 at 13:56, Jingsong Li wrote: > Hi, > > I'm also curious a

Re: Support local aggregate push down for Blink batch planner

2021-01-06 Thread Jark Wu
he interface of SupportsAggregatePushDown can be: interface SupportsAggregatePushDown { boolean applyAggregates( List groupingSets, List aggregates, DataType producedDataType); } What do you think? Best, Jark On Wed, 6 Jan 2021 at 19:56, Sebastian Liu wrote: > Hi Jingsong, Jark, >

Re: Support local aggregate push down for Blink batch planner

2021-01-06 Thread Jark Wu
Thanks for updating the design doc. It looks good to me. Best, Jark On Thu, 7 Jan 2021 at 10:16, Jingsong Li wrote: > Sounds good to me. > > We don't have to worry about future changes, because it has covered all > the capabilities of calcite aggregation. > > Best, >

Re: Support local aggregate push down for Blink batch planner

2021-01-07 Thread Jark Wu
SupportsAggregatePushDown for JDBC source 4. ... Best, Jark On Thu, 7 Jan 2021 at 23:27, Sebastian Liu wrote: > Hi Jark, > > Seems that we have reached the agreement on the proposal. Could you > please help to assign the below jira ticket to me? > https://issues.apache.org/jira/bro

Re: Support local aggregate push down for Blink batch planner

2021-01-08 Thread Jark Wu
Great! Thanks for pushing this work. Looking forward to the pull requests. Best, Jark On Fri, 8 Jan 2021 at 17:57, Sebastian Liu wrote: > Hi Jark, > > Cool, following your suggestions I have created three related subtasks > under Flink-20791. > Hope to assign these subtasks to m

Re: [ANNOUNCE] Welcome Danny Cranmer as a new Apache Flink Committer

2021-01-12 Thread Jark Wu
Congratulations Danny! Best, Jark On Tue, 12 Jan 2021 at 17:59, Yangze Guo wrote: > Congrats, Danny! > > Best, > Yangze Guo > > On Tue, Jan 12, 2021 at 5:55 PM Xingbo Huang wrote: > > > > Congratulations, Danny! > > > > Best, > >

Re: [DISCUSS] FLIP-157 Migrate Flink Documentation from Jekyll to Hugo

2021-01-13 Thread Jark Wu
The build time sounds impressive. Could you explain more what strong internationalization features it provides? Best, Jark On Thu, 14 Jan 2021 at 01:01, Ufuk Celebi wrote: > +1 to do this. I really like what you have build and the advantages to > Jekyll seem overwhelming to me. Hugo i

Re: [VOTE] Release 1.12.1, release candidate #2

2021-01-14 Thread Jark Wu
, nothing unexpected Best, Jark On Thu, 14 Jan 2021 at 12:11, Yang Wang wrote: > +1 (non-binding) > > * Check the checksum and signatures > * Build from source successfully > * Deploy Flink on K8s natively with HA mode and check the related fixes > * FLINK-20650, flink binary c

Re: [DISCUSS] Planning Flink 1.13

2021-01-14 Thread Jark Wu
Thanks for starting the 1.13 release cycle and volunteering as the release managers! The feature freeze date sounds good to me. Best, Jark On Thu, 14 Jan 2021 at 18:19, Khachatryan Roman wrote: > Thanks for doing this! > > Some teams are currently doing (or already finalizing) the &

Re: [VOTE] Release flink-connector-dynamodb v3.0.0, release candidate #0

2022-12-02 Thread Jark Wu
+1 (binding) - Build and compile the source code locally: *OK* - Verified signatures and hashes: *OK* - Checked no missing artifacts in the staging area: *OK* - Reviewed the website release PR: *OK* - Checked the licenses: *OK* Best, Jark On Fri, 2 Dec 2022 at 23:21, Robert Metzger wrote

Re: [VOTE] FLIP-273: Improve Catalog API to Support ALTER TABLE syntax

2022-12-04 Thread Jark Wu
+1 (binding) Best, Jark On Fri, 2 Dec 2022 at 10:11, Paul Lam wrote: > +1 (non-binding) > > Best, > Paul Lam > > > 2022年12月2日 09:17,yuxia 写道: > > > > +1 (non-binding) > > > > Best regards, > > Yuxia > > > > - 原始邮件 - >

Re: [DISCUSS] FLIP-277: Native GlueCatalog Support in Flink

2022-12-09 Thread Jark Wu
t;mydb" under namespace "ns1", is that mean the database name in Flink is "ns1.mydb"? Best, Jark [1]: https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/catalog/MySqlCatalog.java [2]: https://

Re: [VOTE] FLIP-277: Glue catalog support in Flink

2022-12-16 Thread Jark Wu
+1 (binding) It would be better to mention in the FLIP which repository and module to put the Glue Catalog. Best, Jark On Fri, 16 Dec 2022 at 18:18, Samrat Deb wrote: > Hi All, Thanks for all the feedback so far. Based on the discussion[1] we > seem to have a consensus, so I would l

Re: [VOTE] FLIP-275: Support Remote SQL Client Based on SQL Gateway

2022-12-21 Thread Jark Wu
+1 (binding) Best, Jark On Wed, 21 Dec 2022 at 16:00, godfrey he wrote: > +1 (binding) > > Best, > Godfrey > > Hang Ruan 于2022年12月21日周三 15:21写道: > > > > +1 (non-binding) > > > > Best, > > Hang > > > > Paul Lam 于2022年12月20日周二 17:3

Re: [DISCUSS] FLIP-274 : Introduce metric group for OperatorCoordinator

2022-12-29 Thread Jark Wu
roposed Changes" section or "Public Interface" section. As the FLIP said, OperatorCoordinator is widely used in many connectors. Though it is still an @Internal API, I think it is worth mentioning the change in the FLIP. Best, Jark On Mon, 26 Dec 2022 at 18:06, Hang Ruan wrote: &g

Re: [DISCUSS] FLIP-274 : Introduce metric group for OperatorCoordinator

2023-01-03 Thread Jark Wu
declaring that we will implement the interface instead of a no-op method in this FLIP. Anyway, this is a minor problem and shouldn’t block this FLIP. I’m +1 to start a vote. Best, Jark > 2023年1月3日 10:03,Hang Ruan 写道: > > Hi, Jark and Dong, > > Thanks for your comments. Sorry fo

Re: [DISCUSS] Extending the feature freezing date of Flink 1.17

2023-01-03 Thread Jark Wu
+1 for extending the feature freeze date. Best, Jark > 2023年1月3日 19:13,Xingbo Huang 写道: > > +1 for extending the freeze date to Jan 31st > > Best, > Xingbo > > Jingsong Li 于2023年1月3日周二 17:58写道: > >> +1 >> >> On Tue, Jan 3, 2023 at 4:4

Re: [DISCUSS] Adding a option for planner to decide which join reorder rule to choose

2023-01-03 Thread Jark Wu
ld be "table.optimizer.bushy-join-reorder-threshold”? (I guess they are just typos, as your last email said, but I just want to clarify as it is a public API). Best, Jark > 2023年1月3日 12:53,Benchao Li 写道: > > Hi Yunhong, > > Thanks for driving this~ > > I haven'

Re: CDC from Oracle database reading directly logs - integration with OpenLogReplicator

2023-01-04 Thread Jark Wu
you discuss this with the Debezium community? I think Debezium might be interested in this project as well. Best, Jark > 2023年1月5日 07:32,Adam Leszczyński 写道: > > H Márton, > > Thank you very much for your answer. > > The point with Kafka makes sense. It offers

[ANNOUNCE] New Apache Flink Committer - Lincoln Lee

2023-01-09 Thread Jark Wu
ookup Join), FLIP-260 (TableFunction Finish). Besides, He also contributed much to Streaming Semantics, including the non-determinism problem and the message ordering problem. Please join me in congratulating Lincoln for becoming a Flink committer! Cheers, Jark Wu

Re: [DISCUSS] Adding a option for planner to decide which join reorder rule to choose

2023-01-10 Thread Jark Wu
+1 for the option and the default value. Best, Jark > 2023年1月10日 16:24,yh z 写道: > > Hi all, > > Thanks for yours reply. > > After receiving your comments and making targeted modifications. The > conclusion is that the option > "table.optimizer.bushy-join

Re: [VOTE] FLIP-282: Introduce Delete & Update API

2023-01-10 Thread Jark Wu
+1 (binding) Best, Jark > 2023年1月10日 17:39,Lincoln Lee 写道: > > +1 (binding) > > Best, > Lincoln Lee > > > Jingsong Li 于2023年1月10日周二 09:56写道: > >> +1 binding >> >> On Mon, Jan 9, 2023 at 6:14 PM Samrat Deb wrote: >>&

Re: [VOTE] FLIP-274: Introduce metric group for OperatorCoordinator

2023-01-10 Thread Jark Wu
+1 (binding) Best, Jark > 2023年1月10日 12:02,Qingsheng Ren 写道: > > Thanks for the FLIP! > > +1 (binding) > > Best, > Qingsheng > > On Wed, Jan 4, 2023 at 10:08 AM Hang Ruan wrote: > >> Hi all, >> >> Thanks for all the feedback so fa

Re: [VOTE] FLIP-280: Introduce EXPLAIN PLAN_ADVICE to provide SQL advice

2023-01-10 Thread Jark Wu
+1 (binding) Thanks for the detailed design! Best, Jark > 2023年1月10日 17:38,Lincoln Lee 写道: > > +1 (binding) > > Best, > Lincoln Lee > > > godfrey he 于2023年1月10日周二 15:12写道: > >> +1 (binding) >> >> Best, >> Godfrey >>

Re: [VOTE] FLIP-274: Introduce metric group for OperatorCoordinator

2023-01-11 Thread Jark Wu
FLIP indefinitely. Either having a clear waiting time or continuing the development of the FLIP. In the end, we still have a chance to revisit the FLIP before the 1.17 release if Chesnay comes back to the community. Best, Jark On Wed, 11 Jan 2023 at 16:17, Martijn Visser wrote: > -1 (binding) c

Re: [VOTE] FLIP-274: Introduce metric group for OperatorCoordinator

2023-01-11 Thread Jark Wu
or someone to vote for the FLIP who doesn't care about the FLIP for more than a month. Best, Jark [1]: https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws On Wed, 11 Jan 2023 at 21:14, Martijn Visser wrote: > Hi Jark and Dong, > > I fully understand your concerns in th

Re: [VOTE] FLIP-274: Introduce metric group for OperatorCoordinator

2023-01-12 Thread Jark Wu
ching the improved FLIP process. I think this is a very good starting point for discussion. What about starting a new discussion thread for the FLIP process? This thread is a little over divergence. Best, Jark On Thu, 12 Jan 2023 at 14:32, Xintong Song wrote: > Hi all, > > AFAICS, there’

Re: [VOTE] FLIP-279 Unified the max display column width for SqlClient and Table APi in both Streaming and Batch execMode

2023-01-12 Thread Jark Wu
+1 (binding) Thank you for driving this effort. Best, Jark > 2023年1月9日 15:46,Jing Ge 写道: > > Hi, > > I'd like to start a vote on FLIP-279 Unified the max display column width > for SqlClient and Table APi in both Streaming and Batch execMode. The > discussion can

Re: [DISCUSS] Enabling dynamic partition discovery by default in Kafka source

2023-01-12 Thread Jark Wu
+1 for the change. I think this is beneficial for users and is compatible. Best, Jark On Fri, 13 Jan 2023 at 14:22, 何军 wrote: > > > > +1 for this idea, we have enabled kafka dynamic partition discovery in > all > > jobs. > > > > >

Re: [VOTE] Apache Flink Table Store 0.3.0, release candidate #1

2023-01-13 Thread Jark Wu
1.14.5 and 1.15.1 using * Verified web UI and log output, nothing unexpected Best, Jark On Thu, 12 Jan 2023 at 20:52, Jingsong Li wrote: > Thanks Yu, I have created table-store-0.3.1 and move these jiras to 0.3.1. > > Best, > Jingsong > > On Thu, Jan 12, 2023 at 7:

Re: [DISCUSS] FLIP-264 Extract BaseCoordinatorContext

2023-01-16 Thread Jark Wu
rt of DataStream API (e.g., DataStream#dynamicShuffle)? It's easy to extract the common part between SourceCoordinatorContext and ShuffleCoordinatorContext in a single repository as an internal implementation. Best, Jark [1]: https://github.com/apache/hudi/blob/a80bb4f717ad8a89770176a1238c4b

Re: [Discuss] Conventions on driving FLIPs towards consensus

2023-01-27 Thread Jark Wu
conflicts in the future. Best, Jark On Sat, 28 Jan 2023 at 12:26, Xintong Song wrote: > Hi devs, > > Recently, a discussion about how to drive FLIPs towards consensus has taken > place on the private@ mailing list. While many PMC members have already > shared their opinions, we believe

Re: [DISCUSS] FLIP-264 Extract BaseCoordinatorContext

2023-01-27 Thread Jark Wu
Thank Steven for the explanation. It sounds good to me to implement the shuffle operator in the Iceberg project first. We can contribute it to Flink DataStream in the future if other projects/connectors also need it. Best, Jark On Wed, 18 Jan 2023 at 02:11, Steven Wu wrote: > Jark, >

Re: [DISCUSS] FLIP-274 : Introduce metric group for OperatorCoordinator

2023-01-28 Thread Jark Wu
oordinator metrics are just like JMJobMetricGroup, which don't belong to any tasks/vertexes. Do you mean reporting coordinator metrics for every task id under the operator? Best, Jark On Thu, 19 Jan 2023 at 17:33, Chesnay Schepler wrote: > > First, I do not understand why users have to co

Re: [DISCUSS] FLIP-274 : Introduce metric group for OperatorCoordinator

2023-01-30 Thread Jark Wu
s will be exploded when task parallelism is large. This is an overhead to JM & metric systems, and the duplicated metrics are useless for users. 2) Correct me if I'm wrong, MetricGroup#counter only supports registering one metric. It seems it can't register a metric reporting for multi

Re: [DISCUSS] streaming shuffle to improve data clustering and tame small files problem

2023-01-30 Thread Jark Wu
Thank Steven, for starting this discussion. As I suggested in the previous thread, this can be a joint effort beneficial for various projects. I would also like to hear opinions from @Jingsong Li , who is maintaining Flink Table Store. Best, Jark On Tue, 31 Jan 2023 at 08:46, Steven Wu wrote

Re: [DISCUSS] FLIP-274 : Introduce metric group for OperatorCoordinator

2023-01-31 Thread Jark Wu
d by "metrics.scope.jm-job". Similarly, "metrics.scope.tm.job" is replaced with "metrics.scope.tm-job". That's why I though "metrics.scope.jm-operator" might be more consistent. > We would only support operator_name, operator_id, task_name and task_id

Re: [VOTE] FLIP-274: Introduce metric group for OperatorCoordinator

2023-02-02 Thread Jark Wu
+1 Best, Jark On Thu, 2 Feb 2023 at 21:31, Hang Ruan wrote: > Hi all, > > Thanks for all the help about this FLIP. Now let's start the vote again. > Based on the discussion[1], we have come to a consensus, so I would like to > start a vote on FLIP-274: Introd

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

2023-02-06 Thread Jark Wu
g and deprecating APIs, especially when there are known migration gaps. I think it's a good idea to migrate Flink-owned connectors before marking old API deprecated. This ensures downstream projects can migrate to new APIs smoothly. Best, Jark [1]: https://docs.oracle.com/javase/8/do

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

2023-02-06 Thread Jark Wu
new APIs in externalized connectors and giving users the confidence to migrate. Best, Jark [1]: https://lists.apache.org/thread/r5zqnkt01x2c611brkjmxxnt3bfcgl1b On Tue, 7 Feb 2023 at 09:53, yuxia wrote: > Hi Konstantin, > Just FYI, the FileSystemTableSink are still using SinkFunction.

Re: [Discuss] :Introduce Catalog dynamic registration in flink catalog manager.

2023-02-06 Thread Jark Wu
ng path, String className, List resourceUris)". Can this approach satisfy your requirement? Best, Jark On Mon, 6 Feb 2023 at 22:53, Timo Walther wrote: > Hi Feng, > > this is indeed a good proposal. > > 1) It makes sense to improve the catalog listing for platform providers. >

Re: [Discuss] :Introduce Catalog dynamic registration in flink catalog manager.

2023-02-09 Thread Jark Wu
guarantees data consistency and avoids conflicts. This approach can address another pain point of Flink SQL: the catalog information is not persisted. Can this approach satisfy your requirements? Best, Jark On Fri, 10 Feb 2023 at 11:21, Shengkai Fang wrote: > Hi Feng. > > I think your

Re: [Discuss] :Introduce Catalog dynamic registration in flink catalog manager.

2023-02-10 Thread Jark Wu
Thank you Feng, Feel free to start a FLIP proposal if you are interested. Looking forward to it! Best, Jark > 2023年2月10日 15:44,Feng Jin 写道: > > @Shengkai >> About the catalog jar hot updates > > Currently we do not have a similar requirement, but if the catalog >

Re: [Discuss] :Introduce Catalog dynamic registration in flink catalog manager.

2023-02-10 Thread Jark Wu
I have assigned permission to you. Best, Jark > 2023年2月10日 17:26,Feng Jin 写道: > > I am very happy to do it, please help me to add editing permission, my > jira id is hackergin > > Thanks > > Best, > Feng > > On Fri, Feb 10, 2023 at 4:02 PM Jark

Re: [ANNOUNCE] New Apache Flink Committer - Jing Ge

2023-02-14 Thread Jark Wu
Welcome and congratulations, Jing! Best, Jark > 2023年2月14日 23:14,Lijie Wang 写道: > > Congratulations, Jing Ge ! > > Best, > Lijie > > Sergey Nuyanzin 于2023年2月14日周二 21:47写道: > >> Congratulations, Jing Ge! >> >> On Tue, Feb 14, 2023 at 2:47 PM

Re: [ANNOUNCE] New Apache Flink PMC Member - Dong Lin

2023-02-16 Thread Jark Wu
Congratulations, Dong! Best, Jark > 2023年2月16日 20:13,Yanfei Lei 写道: > > Congratulations, Dong! > > Best regards, > Yanfei > > Sergey Nuyanzin 于2023年2月16日周四 17:21写道: >> >> Congratulations, Dong! >> >> On Thu, Feb 16, 2023 at 10:01 AM

Re: [DISCUSS] Proposal to improve Auxiliary Sql Statements

2023-02-21 Thread Jark Wu
Hi Ran, I think it’s always nice to support new syntax if it’s useful for the users. From my side, your syntax table is broken. Could you share it with a Google doc or create a JIRA issue? Best, Jark > 2023年2月21日 17:51,Ran Tao 写道: > > Hi guys. When I recently used flink sql

Re: [DISCUSS] Proposal to improve Auxiliary Sql Statements

2023-02-21 Thread Jark Wu
Thank you, I think this is worth a FLIP design doc to discuss the detailed syntax. Could you prepare a FLIP for that? Best, Jark On Tue, 21 Feb 2023 at 19:37, Ran Tao wrote: > Hi, Jark. thanks. I have added a google doc. > > > https://docs.google.co

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-23 Thread Jark Wu
g methods of accessing and manipulating metadata, they can be extracted to a separate interface, where we can add richer methods. This work can be aligned with the CatalogManager interface (FLIP-295) [2]. Best, Jark [1]: https://stackoverflow.com/questions/7568819/why-no-tail-or-head-method-in

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-25 Thread Jark Wu
l and information schema requires more complex design and discussion, it deserves a separate FLIP. Sergey, are you willing to contribute this FLIP? Best, Jark [1]: https://docs.databricks.com/sql/language-manual/sql-ref-information-schema.html On Fri, 24 Feb 2023 at 22:43, Ran Tao wrote: > Tha

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-28 Thread Jark Wu
Hi Ran, Could you add descriptions about what’s the behavior and differences between the LIKE and ILIKE? Besides, I don’t see the SHOW CONNECTOR syntax and description and how it works in the FLIP. Is it intended to be included in this FLIP? Best, Jark > 2023年2月28日 10:58,Ran Tao

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-28 Thread Jark Wu
Besides, if we introduce the ILIKE, we should also add this feature for the previous SHOW with LIKE statements. They should be included in this FLIP. Best, Jark > 2023年2月28日 17:40,Jark Wu 写道: > > Hi Ran, > > Could you add descriptions about what’s the behavior and differences

Re: [DISCUSS] FLIP-296: Watermark options for table API & SQL

2023-03-01 Thread Jark Wu
rently, the APIs are scattered in the long design sections. Best, Jark [1]: https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals [2]: https://cwiki.apache.org/confluence/display/FLINK/FLIP+Template On Wed, 1 Mar 2023 at 16:56, Kui Yuan wrote: > Hi all, > >

Re: [DISCUSS] FLIP-296: Watermark options for table API & SQL

2023-03-01 Thread Jark Wu
Sorry, I forgot to remind you that Timo's concern about the changes to the CompiledPlan looks like is still not covered in the FLIP. Best, Jark On Wed, 1 Mar 2023 at 19:28, Jark Wu wrote: > Hi Kui, > > Thank you for the great proposal, I think this is already in a good shape. &

Re: [VOTE] Flink minor version support policy for old releases

2023-03-01 Thread Jark Wu
+1 (binding) Best, Jark > 2023年3月2日 05:03,Gyula Fóra 写道: > > +1 (binding) > > Gyula > > On Wed, Mar 1, 2023 at 9:57 PM Thomas Weise wrote: > >> +1 (binding) >> >> Thanks, >> Thomas >> >> On Tue, Feb 28, 2023 at 6:53 AM Sergey

Re: [DISCUSS] FLIP-296: Watermark options for table API & SQL

2023-03-03 Thread Jark Wu
> I'm sorry, perhaps I don't understand what are your concerns > about CompiledPlan. You can consider CompiledPlan and the JSON representation is also a public API that should be backward compatible, e.g., COMPILE PLAN + EXECUTE PLAN should work, and execute a previous plan

Re: [DISCUSS] String literal behavior in Flink

2023-03-05 Thread Jark Wu
with backslash escape string. Best, Jark [1]: https://www.postgresql.org/docs/current/sql-syntax-lexical.html#SQL-SYNTAX-CONSTANTS [2]: https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/overview/ On Sat, 4 Mar 2023 at 23:31, Aitozi wrote: > Hi, > I encoun

Re: Large schemas lead to long DataStream-to-table transformation names

2023-03-05 Thread Jark Wu
Hi Xingcan, I think `physicalDataType.toString()` is indeed verbose in this case. Normal table scan generates descriptions using field names instead of the full schema. Will that help in your case? Best, Jark On Sat, 4 Mar 2023 at 06:57, Xingcan Cui wrote: > Hi all, > > We are dea

Re: [DISCUSS] String literal behavior in Flink

2023-03-05 Thread Jark Wu
Hi Aitozi, I would suggest trying to contribute it to the upstream project Calcite first. Best, Jark > 2023年3月6日 11:51,Aitozi 写道: > > Hi Jark, > > Thank you for your helpful suggestion. It appears that 'E'foo\n'' is a more > versatile and widely accepte

Re: [DISCUSS] String literal behavior in Flink

2023-03-07 Thread Jark Wu
Thank you for sharing this good news! I think we can create an issue for upgrading to Calcite 1.33 and mention the waiting feature. Best, Jark > 2023年3月6日 22:16,Aitozi 写道: > > Hi, Jark > >FYI, this feature has already been supported in Calcite 1.33.0 [1]. > There

Re: [VOTE] FLIP-297: Improve Auxiliary Sql Statements

2023-03-07 Thread Jark Wu
+1 (binding) Best, Jark > 2023年3月7日 17:07,Jing Ge 写道: > > +1 > Thanks! > > Best regards, > Jing > > On Tue, Mar 7, 2023 at 9:51 AM ConradJam wrote: > >> +1 (non-binding). >> >> Hang Ruan 于2023年3月7日周二 16:14写道: >> >>> Thank

[ANNOUNCE] New Apache Flink Committer - Yuxia Luo

2023-03-12 Thread Jark Wu
ns on the user mailing list. Please join me in congratulating Yuxia Luo for becoming a Flink Committer! Best, Jark Wu (on behalf of the Flink PMC)

Re: [VOTE] FLIP-300: Add targetColumns to DynamicTableSink#Context to solve the null overwrite problem of partial-insert

2023-03-13 Thread Jark Wu
+1 (binding) Best, Jark > 2023年3月13日 23:25,Aitozi 写道: > > +1 (non-binding) > > Best, > Aitozi > > Jing Ge 于2023年3月13日周一 22:10写道: > >> +1 (binding) >> >> Best Regards, >> Jing >> >> On Mon, Mar 13, 2023 at 1:57 PM Han

Re: [VOTE] FLIP-296: Extend watermark-related features for SQL

2023-03-20 Thread Jark Wu
+1 (binding) Best, Jark > 2023年3月15日 17:42,Yun Tang 写道: > > +1 (binding) > > Thanks Kui for driving this work. > > > Best > Yun Tang > > From: Kui Yuan > Sent: Wednesday, March 15, 2023 16:45 > To: dev@flink.apache.

Re: [DISCUSS] Planning Flink 1.18

2023-04-01 Thread Jark Wu
Thank Konstantin, Qingsheng, Jing, Sergey for volunteering releases managers. Thank Qingsheng for being the release manager for the two consecutive releases, your experience in v1.17 will greatly help the 1.18 release cycle. Feature freeze in the middle of July sounds good to me. Best, Jark

Re: [DISCUSS] EXACTLY_ONCE delivery semantics for upsert-kafka connector

2023-04-07 Thread Jark Wu
rry, I don’t understand how exactly once can prevent this rollback behavior. Even in your example with EXACTLY_ONCE enabled, the x will go to a5, and b5, then back a5 if jobs perform checkpoints after producing records. Best, Jark > 2023年4月5日 09:39,Alexander Sorokoumov 写道: > > Hello

Re: [DISCUSS] EXACTLY_ONCE delivery semantics for upsert-kafka connector

2023-04-08 Thread Jark Wu
d to expose as few configurations to users as possible. Best, Jark [1] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/group-agg/ On Sat, 8 Apr 2023 at 02:42, Alexander Sorokoumov wrote: > Hi Jark, > > To my knowledge, Kafka's EXACTLY_ONC

Re: [VOTE] FLIP-292: Enhance COMPILED PLAN to support operator-level state TTL configuration

2023-04-10 Thread Jark Wu
+1 (binding) Best, Jark > 2023年4月10日 23:04,godfrey he 写道: > > +1 (binding) > > Best, > Godfrey > > Jing Ge 于2023年4月10日周一 18:42写道: >> >> +1 (binding) >> >> Best Regards, >> Jing >> >> On Mon, Apr 10, 2023 at 12:27 PM Linco

Re: [DISCUSS] EXACTLY_ONCE delivery semantics for upsert-kafka connector

2023-04-12 Thread Jark Wu
ent updates, but we still add the support of 2PC[1]. Best, Jark [1]: https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/jdbc/#jdbcsinkexactlyoncesink On Wed, 12 Apr 2023 at 10:36, John Roesler wrote: > Hi Jark, > > I hope you don’t mind if I chime in. > &g

Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement

2023-04-14 Thread Jark Wu
The FLIP looks good to me. +1 to start a vote. I just have a question: what will happen if the "executeTruncation" returns false without any exceptions? Best, Jark On Thu, 13 Apr 2023 at 19:59, Jing Ge wrote: > Thanks Yuxia for the clarification and FLIP update. The FLIP looks g

Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement

2023-04-14 Thread Jark Wu
Hi Yuxia, Thank you for the updating. That sounds good to me. Best, Jark > 2023年4月14日 19:00,yuxia 写道: > > Hi, Jark. > I'm expecting if the "executeTruncation" returns false, Flink will throw an > generic exception like "Fail to execute truncate ta

[ANNOUNCE] New Apache Flink PMC Member - Leonard Xu

2023-04-21 Thread Jark Wu
dev mailing list, answering questions about 500+ threads in the user/user-zh mailing list. Besides that, he is community minded, such as being the release manager of 1.17, verifying releases, managing release syncs, etc. Congratulations and welcome Leonard! Best, Jark (on behalf of the Flink PMC)

[ANNOUNCE] New Apache Flink PMC Member - Qingsheng Ren

2023-04-21 Thread Jark Wu
. Besides that, he is willing to help a lot in the community work, such as being the release manager for both 1.17 and 1.18, verifying releases, and answering questions on the mailing list. Congratulations and welcome Qingsheng! Best, Jark (on behalf of the Flink PMC)

Re: Re: [VOTE] FLIP-377: Support fine-grained configuration to control filter push down for Table/SQL Sources

2024-01-18 Thread Jark Wu
+1 (binding) Best, Jark On Tue, 16 Jan 2024 at 18:01, Xuyang wrote: > +1 (non-binding) > > > -- > > Best! > Xuyang > > > > > > 在 2024-01-16 17:52:38,"Leonard Xu" 写道: > >+1 (binding) > > > >Best, > >Le

[DISCUSS] Add "Special Thanks" Page on the Flink Website

2024-03-05 Thread Jark Wu
Looking forward to your feedback about this proposal! Best, Jark [1]: https://flink.apache.org/ [2]: https://www.apache.org/foundation/marks/linking#projectthanks [3]: https://hbase.apache.org/sponsors.html [4]: https://mina.apache.org/special-thanks.html [5]: https://cwiki.apache.org/

Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website

2024-03-05 Thread Jark Wu
Sorry, I posted the wrong [7] link. The Flink benchmark ML link is: https://lists.apache.org/thread/bkw6ozoflgltwfwmzjtgx522hyssfko6 On Tue, 5 Mar 2024 at 16:56, Jark Wu wrote: > Hi all, > > > > I want to propose adding a "Special Thanks" page to our Apache Flink we

Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website

2024-03-05 Thread Jark Wu
why I started this discussion to collect the full list. The thank Page is not intended to thank working hours or contributions from individual volunteers which I think is recognized in other ways (e.g., credit of committer and PMC member). Best, Jark [1]: https://www.apache.org/foundation/marks/li

Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website

2024-03-06 Thread Jark Wu
ation, AWS, @Danny Cranmer @HongTeoh! We should add it to the Thank Page!) Best, Jark [1]: https://www.apache.org/foundation/marks/linking#projectthanks [2]: https://cwiki.apache.org/confluence/display/FLINK/Continuous+Integration#ContinuousIntegration-Contacts [3]: https://cwiki.apache

Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website

2024-03-12 Thread Jark Wu
Thank you all for your feedback. If there are no other concerns or objections, I'm going to create a pull request to add the Special Thanks page. Further feedback and sponsors to be added are still welcome! Best, Jark On Mon, 11 Mar 2024 at 23:09, Maximilian Michels wrote: >

Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website

2024-03-12 Thread Jark Wu
I have created a JIRA issue and opened a pull request for this: https://github.com/apache/flink-web/pull/725. Best, Jark On Tue, 12 Mar 2024 at 16:56, Jark Wu wrote: > Thank you all for your feedback. If there are no other concerns or > objections, > I'm going to create a pull

<    2   3   4   5   6   7   8   9   10   11   >