Re: [DISCUSS] Flink SQL Syntax for Query/Savepoint Management

2022-04-17 Thread Paul Lam
Sorry for misspelling your name, Shengkai. The autocomple plugin is not very wise. Best, Paul Lam > 2022年4月18日 11:39,Paul Lam 写道: > > Hi Shanghai, > > You’re right. We can only retrieve the job names from the cluster, and > display > them as query names. > > I agree that the meaning word `

Re: [DISCUSS] Flink SQL Syntax for Query/Savepoint Management

2022-04-17 Thread Paul Lam
Hi Shanghai, You’re right. We can only retrieve the job names from the cluster, and display them as query names. I agree that the meaning word `QUERY` is kind of ambiguous. Strictly speaking, DMLs are not queries, but Hive recognizes DMLs as queries too[1]. In general, I think `QUERY` is more

Re: [DISCUSS] Flink SQL Syntax for Query/Savepoint Management

2022-04-17 Thread Shengkai Fang
Hi, Paul. I am just confused that how the client can retrieve the SQL statement from the cluster? The SQL statement has been translated to the jobgraph and submit to the cluster. I think we will not only manage the query statement lifecyle. How about `SHOW JOBS` and it will list the Job ID, Job N

Re: [DISCUSS] Flink SQL Syntax for Query/Savepoint Management

2022-04-11 Thread Paul Lam
Hi Jark, Thanks a lot! I’m thinking of the 2nd approach. With this approach, the query lifecycle statements (show/stop/savepoint etc) are basically equivalent alternatives to Flink CLI from the user point of view. BTW, the completed jobs might be missing in `SHOW QUERIES`, because for applic

Re: [DISCUSS] Flink SQL Syntax for Query/Savepoint Management

2022-04-10 Thread Jark Wu
Hi Paul, I grant the permission to you. Regarding the "SHOW QUERIES", how will you bookkeep and persist the running and complete queries? Or will you retrieve the queries information from the cluster every time when you receive the command? Best, Jark On Wed, 6 Apr 2022 at 11:23, Paul Lam wro

Re: [DISCUSS] Flink SQL Syntax for Query/Savepoint Management

2022-04-05 Thread Paul Lam
Hi Timo, Thanks for you reply! > It would be great to further investigate which other commands are required > that would be usually be exeuted via CLI commands. I would like to avoid a > large amount of FLIPs each adding a special job lifecycle command. Okay. I listed only the commands about j

Re: [DISCUSS] Flink SQL Syntax for Query/Savepoint Management

2022-04-04 Thread Timo Walther
Hi Paul, thanks for proposing this. I think in general it makes sense to have those commands in SQL Client. However, this will be a big shift because we start adding job lifecycle SQL syntax. It would be great to further investigate which other commands are required that would be usually be

Re: [DISCUSS] Flink SQL Syntax for Query/Savepoint Management

2022-04-01 Thread Paul Lam
Hi Martjin, > > For any extension on the SQL syntax, there should be a FLIP. I would like > to understand how this works for both bounded and unbounded jobs, how this > works with the SQL upgrade story. Could you create one? Sure. I’m preparing one. Please give me the permission if possible. My

Re: [DISCUSS] Flink SQL Syntax for Query/Savepoint Management

2022-04-01 Thread Martijn Visser
Hi Paul, For any extension on the SQL syntax, there should be a FLIP. I would like to understand how this works for both bounded and unbounded jobs, how this works with the SQL upgrade story. Could you create one? I'm also copying in @Timo Walther and @Jark Wu for their opinion on this. Best r

Re: [DISCUSS] Flink SQL Syntax for Query/Savepoint Management

2022-04-01 Thread Paul Lam
Hi Martijn, Thanks a lot for your input. > Have you already thought on how you would implement this in Flink? Yes, I roughly thought about the implementation: 1. Extending Executor to support job list via ClusterClient. 2. Extending Executor to support savepoint trigger/cancel/remove via JobCli

Re: [DISCUSS] Flink SQL Syntax for Query/Savepoint Management

2022-04-01 Thread Martijn Visser
Hi Paul, Thanks for opening the discussion. I agree that there are opportunities in this area to increase user value. I would say that the syntax should be part of a proposal in a FLIP, because the implementation would actually be the complex part, not so much the syntax :) Especially since this