I agree with Ingo. We can open a new JIRA issue or thread to discuss it. Best, Jark
On Mon, 29 Nov 2021 at 16:26, Ingo Bürk <i...@ververica.com> wrote: > I of course meant DQL, not DDL, but same thing either way. It is akin to > e.g. SHOW TABLES. > > On Mon, Nov 29, 2021 at 9:24 AM Ingo Bürk <i...@ververica.com> wrote: > > > To me this doesn't seem too related to the FLIP – this feature would > > require new DDL, and new DDL should live in Flink itself, not in the SQL > > client. > > > > I don't think this requires a FLIP, necessarily, though. It can probably > > just be discussed on a JIRA issue. > > > > > > Ingo > > > > On Mon, Nov 29, 2021 at 9:19 AM Sergey Nuyanzin <snuyan...@gmail.com> > > wrote: > > > >> Hi Daisy > >> > >> Thank you for the proposal. > >> I like it. > >> It was not discussed within this FLIP. > >> > >> I'm not sure about the procedure in such cases (since voting finished). > >> Maybe a separate discussion thread makes sense. > >> > >> However I think it could be done as well if there is no objection. > >> > >> > >> > >> On Fri, Nov 26, 2021 at 2:22 PM Daisy Tsang <da...@ververica.com> > wrote: > >> > >> > Hi, > >> > > >> > would a command that shows the available table source/sink factories > be > >> > implemented? Something that can show that a factory is loaded, etc. > >> > > >> > - Daisy > >> > > >> > On Thu, Nov 4, 2021 at 8:30 AM Sergey Nuyanzin <snuyan...@gmail.com> > >> > wrote: > >> > > >> > > I've started a [VOTE] thread for this FLIP > >> > > https://lists.apache.org/thread/f14jjhrscrdv7h6zw6h1k72nfx232qxs > >> > > > >> > > On Wed, Nov 3, 2021 at 2:59 PM Sergey Nuyanzin <snuyan...@gmail.com > > > >> > > wrote: > >> > > > >> > > > Hi Timo, > >> > > > > >> > > > I completely agree it would be great if we can propagate Calcite > >> parser > >> > > > config in > >> > > > the way you have described. As you mentioned we could discuss this > >> > > > when it comes to the implementation. > >> > > > > >> > > > Meanwhile it looks like I can start voting (please correct me if > I'm > >> > > > wrong). > >> > > > I will start it a bit later today > >> > > > > >> > > > On Wed, Nov 3, 2021 at 1:37 PM Timo Walther <twal...@apache.org> > >> > wrote: > >> > > > > >> > > >> Hi Sergey, > >> > > >> > >> > > >> thanks for your explanation. > >> > > >> > >> > > >> Regarding keywords and other info: We should receive the > >> information > >> > > >> from the Flink SQL parser directly. We have added a couple of new > >> > > >> keywords such as WATERMARK or MATCH_RECOGNIZE clauses. SQL92 > would > >> not > >> > > >> help a user understand why a column name needs to be escaped. And > >> in > >> > > >> general, we should not have duplicate code. Let's discuss this > >> when it > >> > > >> comes to the implementation. I'm sure we can propagate the > Calcite > >> > > >> parser config into a nice POJO that the CLI can receive from the > >> > > Executor. > >> > > >> > >> > > >> Regards, > >> > > >> Timo > >> > > >> > >> > > >> > >> > > >> On 03.11.21 11:12, Sergey Nuyanzin wrote: > >> > > >> > Hi 李宇彬, > >> > > >> > > >> > > >> > I think you are right. Thank you very much for the idea. > >> > > >> > I came across MySQL[1] and PostgreSQL[2] prompts and also > >> > > >> > found several interesting features like control symbols to > change > >> > > style, > >> > > >> > showing current property value and different datetime formats. > >> > > >> > > >> > > >> > I have added your proposals and my findings to FLIP's page, > >> please > >> > > have > >> > > >> a > >> > > >> > look. > >> > > >> > > >> > > >> > [1] > https://dev.mysql.com/doc/refman/8.0/en/mysql-commands.html > >> > > >> > [2] > >> > > https://www.postgresql.org/docs/14/app-psql.html#APP-PSQL-PROMPTING > >> > > >> > > >> > > >> > On Wed, Nov 3, 2021 at 2:31 AM 李宇彬 <lixin58...@163.com> wrote: > >> > > >> > > >> > > >> >> Hi Sergey > >> > > >> >> > >> > > >> >> > >> > > >> >> It is a very useful improvement I'm looking forward to. in > >> > addition, > >> > > I > >> > > >> >> think prompt > >> > > >> >> can play a greater role. > >> > > >> >> > >> > > >> >> > >> > > >> >> To help users call commands in expected context, we can get > >> session > >> > > >> >> context > >> > > >> >> (current catalog/db/time) from cli prompt like MySQL, > >> > > >> >> please see details as below: > >> > > >> >> > >> > > >> >> > >> > > >> >> https://issues.apache.org/jira/browse/FLINK-24730 > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> On 11/2/2021 21:09，Sergey Nuyanzin<snuyan...@gmail.com> > wrote： > >> > > >> >> Hi Timo > >> > > >> >> > >> > > >> >> Thank you for your questions. > >> > > >> >> > >> > > >> >> I will answer your questions here and update FLIP's page as > well > >> > > >> >> > >> > > >> >> For example, who is responsible for parsing comments? I guess > >> the > >> > SQL > >> > > >> >> Client and not the Flink SQL parser will take care of this? > >> > > >> >> Yes, you are right. SQL Client is responsible for parsing > here. > >> > > >> >> However it does not validate sql, it only validates that > >> comments, > >> > > >> brackets > >> > > >> >> and quotes are closed and the statement ends with a semicolon. > >> > > >> >> Also under the hood jline's splits input into words and works > >> with > >> > > >> them. > >> > > >> >> Within a custom parser it is possible to specify what should > be > >> > > >> considered > >> > > >> >> as a word or not considered at all (e.g. it is possible to > >> remove > >> > > >> >> all line and block comments before submitting a query, > >> > > >> >> probably as another non default option)... During parsing it > >> marks > >> > > >> >> what is a comment, a keyword, a quoted string etc. based on > >> rules > >> > > >> >> defined in SQL Client parser. > >> > > >> >> SQL Client highlighter could use the result of this marking to > >> > > >> highlight. > >> > > >> >> Completer could use it to do completion, e.g. if based on > >> parser's > >> > > >> marks > >> > > >> >> completer knows that a cursor is inside a comment or a string > >> > > >> >> then no need to complete anything. > >> > > >> >> > >> > > >> >> Also, will the prompt hints for `'>` and ``>` support > escaping? > >> > This > >> > > >> can > >> > > >> >> be a tricky topic sometimes. > >> > > >> >> Ideally yes, I played with lots of tricky cases and it behaves > >> ok. > >> > > >> >> At least I do not see limitations here. > >> > > >> >> In case you do please share... > >> > > >> >> > >> > > >> >> In general, how do we deal with different SQL dialects in the > >> SQL > >> > > >> >> Client. For example, it possible to `table.sql-dialect` to > >> `HIVE`. > >> > > Will > >> > > >> >> all highlighting, auto-complete and prompt hints be disabled > in > >> > this > >> > > >> case? > >> > > >> >> It could be turned off for the beginning. > >> > > >> >> To make it supported across different dialects it is required > to > >> > have > >> > > >> such > >> > > >> >> info: > >> > > >> >> 1) Set of keywords > >> > > >> >> 2) Quote sign > >> > > >> >> 3) SQL identifier quote > >> > > >> >> 4) Start of a line comment > >> > > >> >> 5) Start and end of a block comment > >> > > >> >> 6) Start and end of hints > >> > > >> >> I see at least 2 ways: > >> > > >> >> 1. provide such api > >> > > >> >> 2. create this mapping in SQL Client and use it based on > current > >> > > >> dialect > >> > > >> >> Then it will be easy to support it for a new dialect > >> > > >> >> Here the only questionable thing is keywords. > >> > > >> >> Currently I made it pretty straightforward: > >> > > >> >> if a word not inside quoted string, not inside a comment or a > >> hint > >> > > >> >> and matches anything from > >> > > >> >> SQL92 ( > >> > > >> >> > >> > > >> >> > >> > > >> > >> > > > >> > > >> > *org.apache.calcite.sql.parser.SqlAbstractParserImpl#getSql92ReservedWords*)), > >> > > >> >> > >> > > >> >> then it will be highlighted as a keyword. > >> > > >> >> > >> > > >> >> On Tue, Nov 2, 2021 at 12:09 PM Timo Walther < > >> twal...@apache.org> > >> > > >> wrote: > >> > > >> >> > >> > > >> >> Hi Sergey, > >> > > >> >> > >> > > >> >> thanks for this nice demo video. It looks very nice and makes > >> the > >> > SQL > >> > > >> >> Client an even more useful tool. > >> > > >> >> > >> > > >> >> What I miss a bit in the FLIP is the implementation details. > >> > > >> >> > >> > > >> >> For example, who is responsible for parsing comments? I guess > >> the > >> > SQL > >> > > >> >> Client and not the Flink SQL parser will take care of this? > >> > > >> >> > >> > > >> >> Also, will the prompt hints for `'>` and ``>` support > escaping? > >> > This > >> > > >> can > >> > > >> >> be a tricky topic sometimes. > >> > > >> >> > >> > > >> >> In general, how do we deal with different SQL dialects in the > >> SQL > >> > > >> >> Client. For example, it possible to `table.sql-dialect` to > >> `HIVE`. > >> > > Will > >> > > >> >> all highlighting, auto-complete and prompt hints be disabled > in > >> > this > >> > > >> case? > >> > > >> >> > >> > > >> >> > >> > > >> >> Looking forward to have this in Flink. > >> > > >> >> > >> > > >> >> Thanks, > >> > > >> >> Timo > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> On 02.11.21 08:26, Till Rohrmann wrote: > >> > > >> >> Hi Sergey, > >> > > >> >> > >> > > >> >> I think that after answering/resolving Jark's comments, you > can > >> > > start a > >> > > >> >> [VOTE] thread for this FLIP. The process is described here > [1]. > >> > Once > >> > > >> the > >> > > >> >> FLIP has been accepted, which it should, given the positive > >> > feedback, > >> > > >> you > >> > > >> >> can start working on it by creating the corresponding JIRA > >> tickets > >> > > and > >> > > >> >> then > >> > > >> >> start coding. I hope that there will be a committer familiar > >> with > >> > the > >> > > >> SQL > >> > > >> >> client that can help you with the code review and merging of > the > >> > > code. > >> > > >> >> But > >> > > >> >> again, given the positive feedback, I think there will be a > >> > > volunteer. > >> > > >> >> > >> > > >> >> [1] > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> > >> > > > >> > > >> > https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals > >> > > >> >> > >> > > >> >> Cheers, > >> > > >> >> Till > >> > > >> >> > >> > > >> >> On Tue, Nov 2, 2021 at 4:25 AM Jark Wu <imj...@gmail.com> > >> wrote: > >> > > >> >> > >> > > >> >> Awesome demo, looking forward to these features! > >> > > >> >> > >> > > >> >> I only have a minor comment: could we provide a config to > >> > > >> enable/disable > >> > > >> >> the prompt values? > >> > > >> >> We can also discuss whether we can enable all the new features > >> by > >> > > >> >> default > >> > > >> >> to give them more exposure. > >> > > >> >> > >> > > >> >> Best, > >> > > >> >> Jark > >> > > >> >> > >> > > >> >> On Tue, 2 Nov 2021 at 10:48, JING ZHANG <beyond1...@gmail.com > > > >> > > wrote: > >> > > >> >> > >> > > >> >> Amazing improvements and impressive video. > >> > > >> >> Big +1. > >> > > >> >> > >> > > >> >> Best, > >> > > >> >> JING ZHANG > >> > > >> >> > >> > > >> >> Kurt Young <ykt...@gmail.com> 于2021年11月2日周二 上午9:37写道： > >> > > >> >> > >> > > >> >> Really cool improvements @Sergey. Can't wait to see it happen. > >> > > >> >> > >> > > >> >> Best, > >> > > >> >> Kurt > >> > > >> >> > >> > > >> >> > >> > > >> >> On Tue, Nov 2, 2021 at 1:56 AM Martijn Visser < > >> > mart...@ververica.com > >> > > > > >> > > >> >> wrote: > >> > > >> >> > >> > > >> >> Hi Sergey, > >> > > >> >> > >> > > >> >> I guess you've just set a new standard ;-) I agree with Ingo, > >> these > >> > > >> >> improvements look really good! > >> > > >> >> > >> > > >> >> Best regards, > >> > > >> >> > >> > > >> >> Martijn > >> > > >> >> > >> > > >> >> On Mon, 1 Nov 2021 at 18:23, Ingo Bürk <i...@ververica.com> > >> wrote: > >> > > >> >> > >> > > >> >> Hi Sergey, > >> > > >> >> > >> > > >> >> I think those improvements look absolutely amazing. Thanks for > >> the > >> > > >> >> little > >> > > >> >> video! > >> > > >> >> > >> > > >> >> > >> > > >> >> Best > >> > > >> >> Ingo > >> > > >> >> > >> > > >> >> On Mon, Nov 1, 2021, 17:15 Sergey Nuyanzin < > snuyan...@gmail.com > >> > > >> > > >> >> wrote: > >> > > >> >> > >> > > >> >> Thanks for the feedback Till. > >> > > >> >> > >> > > >> >> Martijn, I have created a short demo showing some of the > >> features > >> > > >> >> mentioned > >> > > >> >> in FLIP. > >> > > >> >> It is available at https://asciinema.org/a/446247?speed=3.0 > >> > > >> >> Could you please tell if it is what you are expecting or not? > >> > > >> >> > >> > > >> >> On Fri, Oct 29, 2021 at 4:59 PM Till Rohrmann < > >> > > >> >> trohrm...@apache.org> > >> > > >> >> wrote: > >> > > >> >> > >> > > >> >> Thanks for creating this FLIP Sergey. I think what you propose > >> > > >> >> sounds > >> > > >> >> like > >> > > >> >> very good improvements for the SQL client. This should make > the > >> > > >> >> client > >> > > >> >> a > >> > > >> >> lot more ergonomic :-) > >> > > >> >> > >> > > >> >> Cheers, > >> > > >> >> Till > >> > > >> >> > >> > > >> >> On Fri, Oct 29, 2021 at 11:26 AM Sergey Nuyanzin < > >> > > >> >> snuyan...@gmail.com> > >> > > >> >> wrote: > >> > > >> >> > >> > > >> >> Hi Martijn, > >> > > >> >> > >> > > >> >> Thank you for your suggestion with POC. > >> > > >> >> Yes I will do that and come back to this thread probably > >> > > >> >> after > >> > > >> >> the > >> > > >> >> weekend > >> > > >> >> > >> > > >> >> On Thu, Oct 28, 2021 at 4:38 PM Martijn Visser < > >> > > >> >> mart...@ververica.com> > >> > > >> >> wrote: > >> > > >> >> > >> > > >> >> Hi Sergey, > >> > > >> >> > >> > > >> >> Thanks for taking the initiative to create a FLIP and > >> > > >> >> propose > >> > > >> >> improvements > >> > > >> >> on the SQL client. All usability improvements on the SQL > >> > > >> >> client > >> > > >> >> are > >> > > >> >> highly > >> > > >> >> appreciated, especially for new users of Flink. Multi-line > >> > > >> >> support > >> > > >> >> is > >> > > >> >> definitely one of those things I've run into myself. > >> > > >> >> > >> > > >> >> I do think it would be quite nice if there would be some > >> > > >> >> kind > >> > > >> >> of > >> > > >> >> POC > >> > > >> >> which > >> > > >> >> could show (some of) the proposed improvements. Is that > >> > > >> >> something > >> > > >> >> that > >> > > >> >> might be easily feasible? > >> > > >> >> > >> > > >> >> Best regards, > >> > > >> >> > >> > > >> >> Martijn > >> > > >> >> > >> > > >> >> On Thu, 28 Oct 2021 at 11:02, Sergey Nuyanzin < > >> > > >> >> snuyan...@gmail.com > >> > > >> >> > >> > > >> >> wrote: > >> > > >> >> > >> > > >> >> Hi all, > >> > > >> >> > >> > > >> >> I want to start a discussion about FLIP-189: SQL Client > >> > > >> >> Usability > >> > > >> >> Improvements. > >> > > >> >> > >> > > >> >> The main changes in this FLIP: > >> > > >> >> > >> > > >> >> - Flink sql client parser improvements so > >> > > >> >> that sql client does not ask for ; inside a quoted > >> > > >> >> string > >> > > >> >> or a > >> > > >> >> comment > >> > > >> >> - use prompt to show what sql client is waiting for > >> > > >> >> - introduce syntax highlighting > >> > > >> >> - improve completion > >> > > >> >> > >> > > >> >> For more detailed changes, please refer to FLIP-189[1]. > >> > > >> >> > >> > > >> >> [1] > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> > >> > > > >> > > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-189%3A+SQL+Client+Usability+Improvements > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> Look forward to your feedback. > >> > > >> >> > >> > > >> >> -- > >> > > >> >> Best regards, > >> > > >> >> Sergey > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> -- > >> > > >> >> Best regards, > >> > > >> >> Sergey > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> -- > >> > > >> >> Best regards, > >> > > >> >> Sergey > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> -- > >> > > >> >> Best regards, > >> > > >> >> Sergey > >> > > >> >> > >> > > >> > > >> > > >> > > >> > > >> > >> > > >> > >> > > > > >> > > > -- > >> > > > Best regards, > >> > > > Sergey > >> > > > > >> > > > >> > > > >> > > -- > >> > > Best regards, > >> > > Sergey > >> > > > >> > > >> > >> > >> -- > >> Best regards, > >> Sergey > >> > > >
