Hi, Jim. > how to pass additional headers when sending REST requests
Could you share what headers do you want to send when using SQL Client? I think there are two cases we need to consider. Please correct me if I am wrong. # Case 1 If users wants to connect to the SQL Gateway with its password, I think the users should type ``` ./sql-client.sh --user xxx --password xxx ``` in the terminal and the OpenSessionRequest should be enough. # Case 2 If users wants to modify the execution config, users should type ``` Flink SQL> SET `<config_name>` = `<config_value>`; ``` in the terminal. The Client can send ExecuteStatementRequest to the Gateway. > As you have FLIPs or PRs, feel free to let me, Jamie, and Alexey know. It would be nice you can join us to finish the feature. I think the modification about the SQL Gateway side is ready to review. Best, Shengkai Jim Hughes <jhug...@confluent.io.invalid> 于2022年11月11日周五 05:19写道： > Hi Yu Zelin, > > I have read through your draft and it looks good. I am new to Flink, so I > haven't learned about everything which needs to be done yet. > > One of the considerations that I'm interested in understanding is how to > pass additional headers when sending REST requests. From looking at the > code, it looks like a custom `OutboundChannelHandlerFactory` could be > created to read additional configuration and set headers. Does that make > sense? > > Thank you very much for sharing the proof of concept code and the > document. As you have FLIPs or PRs, feel free to let me, Jamie, and Alexey > know. We'll be happy to review them. > > Cheers, > > Jim > > On Wed, Nov 9, 2022 at 11:43 PM yu zelin <yuzelin....@gmail.com> wrote: > > > Hi, all > > Sorry for late response. As Shengkai mentioned, Currently I’m working > with > > him on SQL Client, dedicating to implement the Remote Mode of SQL > Client. I > > have written a draft of implementation plan and will write a FLIP about > it > > ASAP. If you are interested in, please take a look at the draft and it’s > > nice if you give me some feedback. > > The doc is at: > > > https://docs.google.com/document/d/14cS4VBSamMUnlM_PZuK6QKLfriUuQU51iqET5oiYy_c/edit?usp=sharing > > > > > 2022年11月7日 11:19，Shengkai Fang <fskm...@gmail.com> 写道： > > > > > > Hi, all. Sorry for the late reply. > > > > > > > Is the gateway mode planned to be supported for SQL Client in 1.17? > > > > Do you have anything you can already share so we can start with your > > work or just play around with it. > > > > > > Yes. @yzl is working on it and he will list the implementation plan > > later and share the progress. I think the change is not very large and I > > think it's not a big problem to finish this in the release-1.17. I will > > join to develop this in the mid of November. > > > > > > Best, > > > Shengkai > > > > > > > > > > > > > > > Jamie Grier <jgr...@apache.org <mailto:jgr...@apache.org>> > > 于2022年11月5日周六 00:48写道： > > >> Hi Shengkai, > > >> > > >> We're doing more and more Flink development at Confluent these days > and > > we're currently trying to bootstrap a prototype that relies on the SQL > > Client and Gateway. We will be using the the components in some of our > > projects and would like to co-develop them with you and the rest of the > > Flink community. > > >> > > >> As of right now it's a pretty big blocker for our upcoming milestone > > that the SQL Client has not yet been modified to talk to the SQL Gateway > > and we want to help with this effort ASAP! We would be even willing to > > take over the work if it's not yet started but I suspect it already is. > > >> > > >> Anyway, rather than start working immediately on the SQL Client and > > adding a the new Gateway mode ourselves we wanted to start a conversation > > with you and see where you're at with things and offer to help. > > >> > > >> Do you have anything you can already share so we can start with your > > work or just play around with it. Like I said, we just want to get > started > > and are very able to help in this area. We see both the SQL Client and > > Gateway being very important for us and have a good team to help develop > it. > > >> > > >> Let me know if there is a branch you can share, etc. It would be much > > appreciated! > > >> > > >> -Jamie Grier > > >> > > >> > > >> On 2022/10/28 06:06:49 Shengkai Fang wrote: > > >> > Hi. > > >> > > > >> > > Is there a possibility for us to get engaged and at least > introduce > > >> > initial changes to support authentication/authorization? > > >> > > > >> > Yes. You can write a FLIP about the design and change. We can > discuss > > this > > >> > in the dev mail. If the FLIP passes, we can develop it together. > > >> > > > >> > > Another question about persistent Gateway: did you have any > specific > > >> > thoughts about it or some draft design? > > >> > > > >> > We don't have any detailed plan about this. But I know Livy has a > > similar > > >> > feature. > > >> > > > >> > Best, > > >> > Shengkai > > >> > > > >> > > > >> > Alexey Leonov-Vendrovskiy <vendrov...@gmail.com <mailto: > > vendrov...@gmail.com>> 于2022年10月27日周四 15:12写道： > > >> > > > >> > > Apologies from the delayed response on my side. > > >> > > > > >> > > I think the authentication module is not part of our plan in 1.17 > > because > > >> > >> of the busy work. I think we'll start the design at the end of > the > > >> > >> release-1.17. > > >> > > > > >> > > > > >> > > Is there a possibility for us to get engaged and at least > introduce > > >> > > initial changes to support authentication/authorization? > > Specifically, > > >> > > changes in the API and in SQL Client. > > >> > > > > >> > > We expect the following authentication flow: > > >> > > > > >> > > On the SQL gateway we want to be able to use a delegation token. > > >> > > SQL client should be able to supply an API key. > > >> > > The SQL Gateway *would not *be submitting jobs on behalf of the > > client. > > >> > > > > >> > > Ideally it would be nice to introduce some interfaces in the SQL > > Gateway > > >> > > that would allow implementing custom authentication and > > authorization. > > >> > > > > >> > > Another question about persistent Gateway: did you have any > specific > > >> > > thoughts about it or some draft design? > > >> > > > > >> > > Thanks, > > >> > > Alexey > > >> > > > > >> > > > > >> > > On Fri, Oct 21, 2022 at 1:13 AM Shengkai Fang <fskm...@gmail.com > > <mailto:fskm...@gmail.com>> wrote: > > >> > > > > >> > >> Sorry for the late response. > > >> > >> > > >> > >> In the next version(Flink 1.17), we plan to support the SQL > Client > > to > > >> > >> submit the statement to the Flink SQL Gateway. The FLINK-29486 > > >> > >> <https://issues.apache.org/jira/browse/FLINK-29486> is the first > > step to > > >> > >> remove the usage of the `Parser` in the client side, which needs > > to read > > >> > >> the table schema during the converting sql node to operation. I > > think the authentication > > >> > >> module is not part of our plan in 1.17 because of the busy work. > I > > think > > >> > >> we'll start the design at the end of the release-1.17. > > >> > >> But could you share more details about the requirements of the > > >> > >> authentication? > > >> > >> - Do you use the kerberos or delegation token or password to do > the > > >> > >> authentication? > > >> > >> - After the authentication, do you need the sql gateway to submit > > the > > >> > >> job on behalf of the client? > > >> > >> - ... > > >> > >> > > >> > >> For detailed implementation, I think Hive and Presto are good > > examples to > > >> > >> dig in. If you have some thoughts about the authentication > module, > > >> > >> please let me know. > > >> > >> > > >> > >> Best, > > >> > >> Shengkai > > >> > >> > > >> > >> Alexey Leonov-Vendrovskiy <vendrov...@gmail.com <mailto: > > vendrov...@gmail.com>> 于2022年10月19日周三 00:37写道： > > >> > >> > > >> > >>> Thank you for the response, Yuxia! > > >> > >>> > > >> > >>> Shengkai, I would like to learn more about nearest and a bit > more > > >> > >>> distant plans about development of the SQL Gateway and the SQL > > Client. > > >> > >>> Do you have a description of the work planned or maybe can share > > general > > >> > >>> thoughts about the Authentication module, or Persistent Gateway. > > >> > >>> How can the authentication part be addressed on the SQL Client > > side? > > >> > >>> > > >> > >>> Regards, > > >> > >>> -Alexey > > >> > >>> > > >> > >>> > > >> > >>> On Wed, Oct 12, 2022 at 11:24 PM yuxia < > > luoyu...@alumni.sjtu.edu.cn <mailto:luoyu...@alumni.sjtu.edu.cn>> > > >> > >>> wrote: > > >> > >>> > > >> > >>>> > In what Flink’s release the connection from SQL Client to the > > Gateway > > >> > >>>> is > > >> > >>>> expected to be added? > > >> > >>>> Flink 1.17 > > >> > >>>> > > >> > >>>> > “Authentication module” (2) and “Persistent Gateway” (4) as > > >> > >>>> possible future work. Were there any recent discussions on > these > > >> > >>>> subjects? > > >> > >>>> No recent discussions on these subjects, but I think it'll come > > in > > >> > >>>> Flink 1.17 > > >> > >>>> > > >> > >>>> > Another related topic: are there ideas around making SQL > > Gateway a > > >> > >>>> multi-tenant > > >> > >>>> component? > > >> > >>>> Yes. > > >> > >>>> > > >> > >>>> Shengkaiis the maintainer of SQL Client and SQL gateway, maybe > > he can > > >> > >>>> provide more information. > > >> > >>>> > > >> > >>>> > > >> > >>>> > > >> > >>>> Best regards, > > >> > >>>> Yuxia > > >> > >>>> > > >> > >>>> ----- 原始邮件 ----- > > >> > >>>> 发件人: "Alexey Leonov-Vendrovskiy" <vendrov...@gmail.com > <mailto: > > vendrov...@gmail.com>> > > >> > >>>> 收件人: "dev" <dev@flink.apache.org <mailto:dev@flink.apache.org > >> > > >> > >>>> 发送时间: 星期四, 2022年 10 月 13日 下午 12:33:08 > > >> > >>>> 主题: SQL Gateway and SQL Client > > >> > >>>> > > >> > >>>> Hi all, > > >> > >>>> > > >> > >>>> I’m Alexey from Confluent. This is my first email in this > > discussion > > >> > >>>> list. > > >> > >>>> I’m rather new to Flink, and to local customs of communication. > > I want > > >> > >>>> to > > >> > >>>> dive deeper and hopefully get more involved over time. > > >> > >>>> > > >> > >>>> Currently I have a few questions around SQL Gateway and SQL > > Client. > > >> > >>>> Specifically I wanted to learn what is the vision around the > > nearest > > >> > >>>> future > > >> > >>>> of these two components. > > >> > >>>> > > >> > >>>> In what Flink’s release the connection from SQL Client to the > > Gateway is > > >> > >>>> expected to be added? I was looking at > > >> > >>>> https://issues.apache.org/jira/browse/FLINK-29486, and > recently > > it got > > >> > >>>> renamed from “Enable SQL Client to Connect SQL Gateway in > Remote > > Mode” > > >> > >>>> to > > >> > >>>> “Introduce Client Parser to get statement type”. I did some > > search, but > > >> > >>>> didn’t find a good place where the client's work in this > > direction is > > >> > >>>> discussed or tracked. > > >> > >>>> > > >> > >>>> A couple questions about the SQL Gateway. The FLIP-91 > > >> > >>>> < > > >> > >>>> > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-91%3A+Support+SQL+Gateway#FLIP91:SupportSQLGateway-Futurework > > >> > >>>> > > > >> > >>>> mentions “Authentication module” (2) and “Persistent Gateway” > > (4) as > > >> > >>>> possible future work. Were there any recent discussions on > these > > >> > >>>> subjects? > > >> > >>>> Or maybe there are some ideas how to move these directions > > forward? > > >> > >>>> Another > > >> > >>>> related topic: are there ideas around making SQL Gateway a > > multi-tenant > > >> > >>>> component? > > >> > >>>> > > >> > >>>> Thank you, > > >> > >>>> > > >> > >>>> Alexey > > >> > >>>> > > >> > >>> > > >> > > > > > >
