Hi Paul, For any extension on the SQL syntax, there should be a FLIP. I would like to understand how this works for both bounded and unbounded jobs, how this works with the SQL upgrade story. Could you create one?
I'm also copying in @Timo Walther <twal...@apache.org> and @Jark Wu <imj...@gmail.com> for their opinion on this. Best regards, Martijn On Fri, 1 Apr 2022 at 12:01, Paul Lam <paullin3...@gmail.com> wrote: > Hi Martijn, > > Thanks a lot for your input. > > > Have you already thought on how you would implement this in Flink? > > Yes, I roughly thought about the implementation: > > 1. Extending Executor to support job list via ClusterClient. > 2. Extending Executor to support savepoint trigger/cancel/remove via > JobClient. > 3. Extending SQL parser to support the new statements via regex > (AbstractRegexParseStrategy) or Calcite. > > IMHO, the implementation is not very complicated and barely touches the > architecture of FLIP-91. > (BTW, FLIP-91 might be a little bit outdated and doesn’t fully reflect > the current status of Flink SQL client/gateway.) > > WDYT? > > Best, > Paul Lam > > > 2022年4月1日 17:33，Martijn Visser <mart...@ververica.com> 写道： > > > > Hi Paul, > > > > Thanks for opening the discussion. I agree that there are opportunities > in > > this area to increase user value. > > > > I would say that the syntax should be part of a proposal in a FLIP, > because > > the implementation would actually be the complex part, not so much the > > syntax :) Especially since this also touches on FLIP-91 [1] > > > > Have you already thought on how you would implement this in Flink? > > > > Best regards, > > > > Martijn Visser > > https://twitter.com/MartijnVisser82 > > https://github.com/MartijnVisser > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-91%3A+Support+SQL+Client+Gateway > > > > > > On Fri, 1 Apr 2022 at 11:25, Paul Lam <paullin3...@gmail.com> wrote: > > > >> Hi team, > >> > >> Greetings from Apache Kyuubi(incubating) community. We’re integrating > >> Flink as a SQL engine and aiming to make it production-ready. > >> > >> However, query/savepoint management is a crucial but missing part in > Flink > >> SQL, thus we reach out to discuss the SQL syntax with Flink community. > >> > >> We propose to introduce the following statements: > >> > >> SHOW QUERIES: shows the running queries in the current session, which > >> mainly returns query(namely Flink job) IDs and SQL statements. > >> TRIGGER SAVEPOINT <query_id>: triggers a savepoint for the specified > >> query, which returns the stored path of the savepoint. > >> SHOW SAVEPOINTS <query_id>: shows the savepoints for the specified > query, > >> which returns the stored paths of the savepoints. > >> REMOVE SAVEPOINT <savepoint_path>: removes the specified savepoint. > >> > >> WRT to keywords, `TRIGGER` and `SAVEPOINT` are already reserved keywords > >> in Flink SQL[1], so the only new keyword is `QUERIES`. > >> > >> If we reach a consensus on the syntax, we could either implement it in > >> Kyuubi and contribute back to Flink, or directly implement it in Flink. > >> > >> Looking forward for your feedback ;) > >> > >> [1] > >> > https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/sql/overview/#reserved-keywords > >> > >> Best, > >> Paul Lam > >> > >> > >
