Re: SQL Gateway and SQL Client

Hi, Jim, Thanks for your feedback, your suggestions are very good. I have replied in the discussion mail, please take a look. > 2022年11月29日 03:18，Jim Hughes 写道： > > Hi Shengkai, Yu, > > Thanks for the FLIP! I have had a chance to read it, and it looks good. I > do have some questions: > >

Re: SQL Gateway and SQL Client

Hi Shengkai, Yu, Thanks for the FLIP! I have had a chance to read it, and it looks good. I do have some questions: I do like the idea of unifying the approaches so that the code doesn't get out of step. 1. For the Client Parser, is it going to work with the extended syntax from the Flink

Re: SQL Gateway and SQL Client

Hi, Jim and Alexey. We have written the proposal[1]. It would be appreciated if you can give us some feedback. Best, Shengkai [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-275%3A+Support+Remote+SQL+Client+Based+on+SQL+Gateway yu zelin 于2022年11月24日周四 12:06写道： > Hi Jim, > Sorry for

Re: SQL Gateway and SQL Client

Hi Jim, Sorry for incorrect message in last reply. > Shengkai will help to your PR I mean Shengkai will help to review the PRs. And I will add some of your suggestion to my design. I think the POC code will be cherry-picked to my new design of SQL Client. > 2022年11月23日 12:18，yu zelin 写道： > > H

Re: SQL Gateway and SQL Client

Hi Jim, Sorry for late response. Just another busy week :) Last week, I’ve discussed within my team about my design. My teammates think it’s better to unify the local and remote mode, so I’ve investigated and redesigned a new plan. I’ll inform you after the rewriting of FLIP finished (will be so

Re: SQL Gateway and SQL Client

Hi Yu, Shengkai, As a quick update, I've had a chance to try out Yu's POC and it is working for me. (Admittedly, I haven't tried it too extensively; I only tried basic operations.) >From my experiments, I did leave a few comments on https://github.com/apache/flink/pull/20958. Overall, the PRs I

Re: SQL Gateway and SQL Client

Hi Shengkai, Thanks! Can you say more about how the RestClient sets the headers? I looked at the code, and I only see headers being set in the RestClient here[1]. I'm hoping to set arbitrary headers from configuration. I'm not seeing a way to do that presently. [1] https://github.com/apache/f

Re: SQL Gateway and SQL Client

Hi Yu, The PR looks good to me; the only thing I noticed is that the RemoveJarOperation may need to be added. I asked some other questions; I don't think any of them are particularly large issues. Let me know when the next PR goes up and I'll take a look. Cheers, Jim On Sun, Nov 13, 2022 at 1

Re: SQL Gateway and SQL Client

Hi Jim, It would be nice if you can take a look on https://github.com/apache/flink/pull/21133 and give me some feedback. Best, Yu Zelin > 2022年11月12日 00:44，Jim Hughes 写道： > > Hi Shengkai, > > I think there is an additional case where a proxy is between the client and > gateway. In that case

Re: SQL Gateway and SQL Client

Hi Jim, Thanks for your input. You can look here[1] on the server side. You can modify the return type of the handleRequest to Tuple2 to get session-specific headers. RestClient has already exposed a method to send requests with headers for the client side. [1] https://github.com/apache/flink/blo

Re: SQL Gateway and SQL Client

Hi Shengkai, I think there is an additional case where a proxy is between the client and gateway. In that case, being able to pass headers would allow for additional options / features. I see several PRs from Yu Zelin. Is there a first one to review? Cheers, Jim On Thu, Nov 10, 2022 at 9:42

Re: SQL Gateway and SQL Client

Hi, Jim. > how to pass additional headers when sending REST requests Could you share what headers do you want to send when using SQL Client? I think there are two cases we need to consider. Please correct me if I am wrong. # Case 1 If users wants to connect to the SQL Gateway with its password

Re: SQL Gateway and SQL Client

Hi Yu Zelin, I have read through your draft and it looks good. I am new to Flink, so I haven't learned about everything which needs to be done yet. One of the considerations that I'm interested in understanding is how to pass additional headers when sending REST requests. From looking at the co

Re: SQL Gateway and SQL Client

Hi, all Sorry for late response. As Shengkai mentioned, Currently I’m working with him on SQL Client, dedicating to implement the Remote Mode of SQL Client. I have written a draft of implementation plan and will write a FLIP about it ASAP. If you are interested in, please take a look at the draf

Re: SQL Gateway and SQL Client

Hi, all. Sorry for the late reply. > Is the gateway mode planned to be supported for SQL Client in 1.17? > Do you have anything you can already share so we can start with your work or just play around with it. Yes. @yzl is working on it and he will list the implementation plan later and share the

Re: SQL Gateway and SQL Client

Hi Shengkai, We're doing more and more Flink development at Confluent these days and we're currently trying to bootstrap a prototype that relies on the SQL Client and Gateway. We will be using the the components in some of our projects and would like to co-develop them with you and the rest of

Re: SQL Gateway and SQL Client

Thanks Shengkai. > We don't have any detailed plan about this. But I know Livy has a similar feature. How can we check with Livy on this? Back to the SQL client: is this the master issue tracking enabling SQL Client <-> Gateway connection? https://issues.apache.org/jira/browse/FLINK-29751 (Migra

Re: SQL Gateway and SQL Client

Hi. > Is there a possibility for us to get engaged and at least introduce initial changes to support authentication/authorization? Yes. You can write a FLIP about the design and change. We can discuss this in the dev mail. If the FLIP passes, we can develop it together. > Another question about

Re: SQL Gateway and SQL Client

Apologies from the delayed response on my side. I think the authentication module is not part of our plan in 1.17 because > of the busy work. I think we'll start the design at the end of the > release-1.17. Is there a possibility for us to get engaged and at least introduce initial changes to s

Re: SQL Gateway and SQL Client

Sorry for the late response. In the next version(Flink 1.17), we plan to support the SQL Client to submit the statement to the Flink SQL Gateway. The FLINK-29486 is the first step to remove the usage of the `Parser` in the client side, which need

Re: SQL Gateway and SQL Client

Thank you for the response, Yuxia! Shengkai, I would like to learn more about nearest and a bit more distant plans about development of the SQL Gateway and the SQL Client. Do you have a description of the work planned or maybe can share general thoughts about the Authentication module, or Persiste

Re: SQL Gateway and SQL Client

> In what Flink’s release the connection from SQL Client to the Gateway is expected to be added? Flink 1.17 > “Authentication module” (2) and “Persistent Gateway” (4) as possible future work. Were there any recent discussions on these subjects? No recent discussions on these subjects, but I think