Hi, Jingsong. Thanks for your reply. I think `UNSET` is much better.
1. We don't need to introduce another command `UNSET`. `RESET` is supported in the current sql client now. Our proposal just extends its grammar and allow users to reset the specified keys. 2. Hive beeline also uses `RESET` to set the key to the default value[1]. I think it is more friendly for batch users. Best, Shengkai [1] https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients Jingsong Li <jingsongl...@gmail.com> 于2021年2月2日周二 下午1:56写道： > Thanks for the proposal, yes, sql-client is too outdated. +1 for improving > it. > > About "SET" and "RESET", Why not be "SET" and "UNSET"? > > Best, > Jingsong > > On Mon, Feb 1, 2021 at 2:46 PM Rui Li <lirui.fu...@gmail.com> wrote: > >> Thanks Shengkai for the update! The proposed changes look good to me. >> >> On Fri, Jan 29, 2021 at 8:26 PM Shengkai Fang <fskm...@gmail.com> wrote: >> >> > Hi, Rui. >> > You are right. I have already modified the FLIP. >> > >> > The main changes: >> > >> > # -f parameter has no restriction about the statement type. >> > Sometimes, users use the pipe to redirect the result of queries to debug >> > when submitting job by -f parameter. It's much convenient comparing to >> > writing INSERT INTO statements. >> > >> > # Add a new sql client option `sql-client.job.detach` . >> > Users prefer to execute jobs one by one in the batch mode. Users can set >> > this option false and the client will process the next job until the >> > current job finishes. The default value of this option is false, which >> > means the client will execute the next job when the current job is >> > submitted. >> > >> > Best, >> > Shengkai >> > >> > >> > >> > Rui Li <lirui.fu...@gmail.com> 于2021年1月29日周五 下午4:52写道： >> > >> >> Hi Shengkai, >> >> >> >> Regarding #2, maybe the -f options in flink and hive have different >> >> implications, and we should clarify the behavior. For example, if the >> >> client just submits the job and exits, what happens if the file >> contains >> >> two INSERT statements? I don't think we should treat them as a >> statement >> >> set, because users should explicitly write BEGIN STATEMENT SET in that >> >> case. And the client shouldn't asynchronously submit the two jobs, >> because >> >> the 2nd may depend on the 1st, right? >> >> >> >> On Fri, Jan 29, 2021 at 4:30 PM Shengkai Fang <fskm...@gmail.com> >> wrote: >> >> >> >>> Hi Rui, >> >>> Thanks for your feedback. I agree with your suggestions. >> >>> >> >>> For the suggestion 1: Yes. we are plan to strengthen the set command. >> In >> >>> the implementation, it will just put the key-value into the >> >>> `Configuration`, which will be used to generate the table config. If >> hive >> >>> supports to read the setting from the table config, users are able to >> set >> >>> the hive-related settings. >> >>> >> >>> For the suggestion 2: The -f parameter will submit the job and exit. >> If >> >>> the queries never end, users have to cancel the job by themselves, >> which is >> >>> not reliable(people may forget their jobs). In most case, queries are >> used >> >>> to analyze the data. Users should use queries in the interactive mode. >> >>> >> >>> Best, >> >>> Shengkai >> >>> >> >>> Rui Li <lirui.fu...@gmail.com> 于2021年1月29日周五 下午3:18写道： >> >>> >> >>>> Thanks Shengkai for bringing up this discussion. I think it covers a >> >>>> lot of useful features which will dramatically improve the usability >> of our >> >>>> SQL Client. I have two questions regarding the FLIP. >> >>>> >> >>>> 1. Do you think we can let users set arbitrary configurations via the >> >>>> SET command? A connector may have its own configurations and we >> don't have >> >>>> a way to dynamically change such configurations in SQL Client. For >> example, >> >>>> users may want to be able to change hive conf when using hive >> connector [1]. >> >>>> 2. Any reason why we have to forbid queries in SQL files specified >> with >> >>>> the -f option? Hive supports a similar -f option but allows queries >> in the >> >>>> file. And a common use case is to run some query and redirect the >> results >> >>>> to a file. So I think maybe flink users would like to do the same, >> >>>> especially in batch scenarios. >> >>>> >> >>>> [1] https://issues.apache.org/jira/browse/FLINK-20590 >> >>>> >> >>>> On Fri, Jan 29, 2021 at 10:46 AM Sebastian Liu < >> liuyang0...@gmail.com> >> >>>> wrote: >> >>>> >> >>>>> Hi Shengkai, >> >>>>> >> >>>>> Glad to see this improvement. And I have some additional >> suggestions: >> >>>>> >> >>>>> #1. Unify the TableEnvironment in ExecutionContext to >> >>>>> StreamTableEnvironment for both streaming and batch sql. >> >>>>> #2. Improve the way of results retrieval: sql client collect the >> >>>>> results >> >>>>> locally all at once using accumulators at present, >> >>>>> which may have memory issues in JM or Local for the big query >> >>>>> result. >> >>>>> Accumulator is only suitable for testing purpose. >> >>>>> We may change to use SelectTableSink, which is based >> >>>>> on CollectSinkOperatorCoordinator. >> >>>>> #3. Do we need to consider Flink SQL gateway which is in FLIP-91. >> Seems >> >>>>> that this FLIP has not moved forward for a long time. >> >>>>> Provide a long running service out of the box to facilitate >> the >> >>>>> sql >> >>>>> submission is necessary. >> >>>>> >> >>>>> What do you think of these? >> >>>>> >> >>>>> [1] >> >>>>> >> >>>>> >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-91%3A+Support+SQL+Client+Gateway >> >>>>> >> >>>>> >> >>>>> Shengkai Fang <fskm...@gmail.com> 于2021年1月28日周四 下午8:54写道： >> >>>>> >> >>>>> > Hi devs, >> >>>>> > >> >>>>> > Jark and I want to start a discussion about FLIP-163:SQL Client >> >>>>> > Improvements. >> >>>>> > >> >>>>> > Many users have complained about the problems of the sql client. >> For >> >>>>> > example, users can not register the table proposed by FLIP-95. >> >>>>> > >> >>>>> > The main changes in this FLIP: >> >>>>> > >> >>>>> > - use -i parameter to specify the sql file to initialize the table >> >>>>> > environment and deprecated YAML file; >> >>>>> > - add -f to submit sql file and deprecated '-u' parameter; >> >>>>> > - add more interactive commands, e.g ADD JAR; >> >>>>> > - support statement set syntax; >> >>>>> > >> >>>>> > >> >>>>> > For more detailed changes, please refer to FLIP-163[1]. >> >>>>> > >> >>>>> > Look forward to your feedback. >> >>>>> > >> >>>>> > >> >>>>> > Best, >> >>>>> > Shengkai >> >>>>> > >> >>>>> > [1] >> >>>>> > >> >>>>> > >> >>>>> >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-163%3A+SQL+Client+Improvements >> >>>>> > >> >>>>> >> >>>>> >> >>>>> -- >> >>>>> >> >>>>> *With kind regards >> >>>>> ------------------------------------------------------------ >> >>>>> Sebastian Liu 刘洋 >> >>>>> Institute of Computing Technology, Chinese Academy of Science >> >>>>> Mobile\WeChat: +86—15201613655 >> >>>>> E-mail: liuyang0...@gmail.com <liuyang0...@gmail.com> >> >>>>> QQ: 3239559* >> >>>>> >> >>>> >> >>>> >> >>>> -- >> >>>> Best regards! >> >>>> Rui Li >> >>>> >> >>> >> >> >> >> -- >> >> Best regards! >> >> Rui Li >> >> >> > >> >> -- >> Best regards! >> Rui Li >> > > > -- > Best, Jingsong Lee >
