Sorry for the typo. I mean `RESET` is much better rather than `UNSET`. Shengkai Fang <fskm...@gmail.com> 于2021年2月2日周二 下午4:44写道：
> Hi, Jingsong. > > Thanks for your reply. I think `UNSET` is much better. > > 1. We don't need to introduce another command `UNSET`. `RESET` is > supported in the current sql client now. Our proposal just extends its > grammar and allow users to reset the specified keys. > 2. Hive beeline also uses `RESET` to set the key to the default value[1]. > I think it is more friendly for batch users. > > Best, > Shengkai > > [1] https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients > > Jingsong Li <jingsongl...@gmail.com> 于2021年2月2日周二 下午1:56写道： > >> Thanks for the proposal, yes, sql-client is too outdated. +1 for >> improving it. >> >> About "SET" and "RESET", Why not be "SET" and "UNSET"? >> >> Best, >> Jingsong >> >> On Mon, Feb 1, 2021 at 2:46 PM Rui Li <lirui.fu...@gmail.com> wrote: >> >>> Thanks Shengkai for the update! The proposed changes look good to me. >>> >>> On Fri, Jan 29, 2021 at 8:26 PM Shengkai Fang <fskm...@gmail.com> wrote: >>> >>> > Hi, Rui. >>> > You are right. I have already modified the FLIP. >>> > >>> > The main changes: >>> > >>> > # -f parameter has no restriction about the statement type. >>> > Sometimes, users use the pipe to redirect the result of queries to >>> debug >>> > when submitting job by -f parameter. It's much convenient comparing to >>> > writing INSERT INTO statements. >>> > >>> > # Add a new sql client option `sql-client.job.detach` . >>> > Users prefer to execute jobs one by one in the batch mode. Users can >>> set >>> > this option false and the client will process the next job until the >>> > current job finishes. The default value of this option is false, which >>> > means the client will execute the next job when the current job is >>> > submitted. >>> > >>> > Best, >>> > Shengkai >>> > >>> > >>> > >>> > Rui Li <lirui.fu...@gmail.com> 于2021年1月29日周五 下午4:52写道： >>> > >>> >> Hi Shengkai, >>> >> >>> >> Regarding #2, maybe the -f options in flink and hive have different >>> >> implications, and we should clarify the behavior. For example, if the >>> >> client just submits the job and exits, what happens if the file >>> contains >>> >> two INSERT statements? I don't think we should treat them as a >>> statement >>> >> set, because users should explicitly write BEGIN STATEMENT SET in that >>> >> case. And the client shouldn't asynchronously submit the two jobs, >>> because >>> >> the 2nd may depend on the 1st, right? >>> >> >>> >> On Fri, Jan 29, 2021 at 4:30 PM Shengkai Fang <fskm...@gmail.com> >>> wrote: >>> >> >>> >>> Hi Rui, >>> >>> Thanks for your feedback. I agree with your suggestions. >>> >>> >>> >>> For the suggestion 1: Yes. we are plan to strengthen the set >>> command. In >>> >>> the implementation, it will just put the key-value into the >>> >>> `Configuration`, which will be used to generate the table config. If >>> hive >>> >>> supports to read the setting from the table config, users are able >>> to set >>> >>> the hive-related settings. >>> >>> >>> >>> For the suggestion 2: The -f parameter will submit the job and exit. >>> If >>> >>> the queries never end, users have to cancel the job by themselves, >>> which is >>> >>> not reliable(people may forget their jobs). In most case, queries >>> are used >>> >>> to analyze the data. Users should use queries in the interactive >>> mode. >>> >>> >>> >>> Best, >>> >>> Shengkai >>> >>> >>> >>> Rui Li <lirui.fu...@gmail.com> 于2021年1月29日周五 下午3:18写道： >>> >>> >>> >>>> Thanks Shengkai for bringing up this discussion. I think it covers a >>> >>>> lot of useful features which will dramatically improve the >>> usability of our >>> >>>> SQL Client. I have two questions regarding the FLIP. >>> >>>> >>> >>>> 1. Do you think we can let users set arbitrary configurations via >>> the >>> >>>> SET command? A connector may have its own configurations and we >>> don't have >>> >>>> a way to dynamically change such configurations in SQL Client. For >>> example, >>> >>>> users may want to be able to change hive conf when using hive >>> connector [1]. >>> >>>> 2. Any reason why we have to forbid queries in SQL files specified >>> with >>> >>>> the -f option? Hive supports a similar -f option but allows queries >>> in the >>> >>>> file. And a common use case is to run some query and redirect the >>> results >>> >>>> to a file. So I think maybe flink users would like to do the same, >>> >>>> especially in batch scenarios. >>> >>>> >>> >>>> [1] https://issues.apache.org/jira/browse/FLINK-20590 >>> >>>> >>> >>>> On Fri, Jan 29, 2021 at 10:46 AM Sebastian Liu < >>> liuyang0...@gmail.com> >>> >>>> wrote: >>> >>>> >>> >>>>> Hi Shengkai, >>> >>>>> >>> >>>>> Glad to see this improvement. And I have some additional >>> suggestions: >>> >>>>> >>> >>>>> #1. Unify the TableEnvironment in ExecutionContext to >>> >>>>> StreamTableEnvironment for both streaming and batch sql. >>> >>>>> #2. Improve the way of results retrieval: sql client collect the >>> >>>>> results >>> >>>>> locally all at once using accumulators at present, >>> >>>>> which may have memory issues in JM or Local for the big query >>> >>>>> result. >>> >>>>> Accumulator is only suitable for testing purpose. >>> >>>>> We may change to use SelectTableSink, which is based >>> >>>>> on CollectSinkOperatorCoordinator. >>> >>>>> #3. Do we need to consider Flink SQL gateway which is in FLIP-91. >>> Seems >>> >>>>> that this FLIP has not moved forward for a long time. >>> >>>>> Provide a long running service out of the box to facilitate >>> the >>> >>>>> sql >>> >>>>> submission is necessary. >>> >>>>> >>> >>>>> What do you think of these? >>> >>>>> >>> >>>>> [1] >>> >>>>> >>> >>>>> >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-91%3A+Support+SQL+Client+Gateway >>> >>>>> >>> >>>>> >>> >>>>> Shengkai Fang <fskm...@gmail.com> 于2021年1月28日周四 下午8:54写道： >>> >>>>> >>> >>>>> > Hi devs, >>> >>>>> > >>> >>>>> > Jark and I want to start a discussion about FLIP-163:SQL Client >>> >>>>> > Improvements. >>> >>>>> > >>> >>>>> > Many users have complained about the problems of the sql client. >>> For >>> >>>>> > example, users can not register the table proposed by FLIP-95. >>> >>>>> > >>> >>>>> > The main changes in this FLIP: >>> >>>>> > >>> >>>>> > - use -i parameter to specify the sql file to initialize the >>> table >>> >>>>> > environment and deprecated YAML file; >>> >>>>> > - add -f to submit sql file and deprecated '-u' parameter; >>> >>>>> > - add more interactive commands, e.g ADD JAR; >>> >>>>> > - support statement set syntax; >>> >>>>> > >>> >>>>> > >>> >>>>> > For more detailed changes, please refer to FLIP-163[1]. >>> >>>>> > >>> >>>>> > Look forward to your feedback. >>> >>>>> > >>> >>>>> > >>> >>>>> > Best, >>> >>>>> > Shengkai >>> >>>>> > >>> >>>>> > [1] >>> >>>>> > >>> >>>>> > >>> >>>>> >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-163%3A+SQL+Client+Improvements >>> >>>>> > >>> >>>>> >>> >>>>> >>> >>>>> -- >>> >>>>> >>> >>>>> *With kind regards >>> >>>>> ------------------------------------------------------------ >>> >>>>> Sebastian Liu 刘洋 >>> >>>>> Institute of Computing Technology, Chinese Academy of Science >>> >>>>> Mobile\WeChat: +86—15201613655 >>> >>>>> E-mail: liuyang0...@gmail.com <liuyang0...@gmail.com> >>> >>>>> QQ: 3239559* >>> >>>>> >>> >>>> >>> >>>> >>> >>>> -- >>> >>>> Best regards! >>> >>>> Rui Li >>> >>>> >>> >>> >>> >> >>> >> -- >>> >> Best regards! >>> >> Rui Li >>> >> >>> > >>> >>> -- >>> Best regards! >>> Rui Li >>> >> >> >> -- >> Best, Jingsong Lee >> >
