Hi Shengkai, Yu, Thanks for the FLIP! I have had a chance to read it, and it looks good. I do have some questions:
I do like the idea of unifying the approaches so that the code doesn't get out of step. 1. For the Client Parser, is it going to work with the extended syntax from the Flink Table Store? 2. Relatedly, what will happen if an older Client tries to handle syntax that a newer service supports? (Suppose I use a 1.17 client with a 1.18 Gateway/system which has a new keyword. Is there anything we should be designing for upfront?) 3. How will client and server version mismatches be handled? Will a single gateway be able to support multiple endpoint versions? 4. How are commands which change a session handled? Are those sent via an ExecuteStatementRequest? 5. The remote POC uses polling for getting back status and getting back results. Would it be possible to switch to web sockets or some other mechanism to avoid polling? If polling is used for both, the polling frequency should be different between local and remote configurations. 6. What does this sentence mean? "The reason why we didn't get the sql type in client side is because it's hard for the lightweight client-level parser to recognize some sql type sql, such as query with CTE. " 7. What is the serialization lifecycle for results? It makes sense to have some control over whether the gateway returns results as SQL or JSON. I'd love to see a way to avoid needing to serialize and deserialize results on the SQL Gateway if possible. I'm still new enough to the project that I'm not sure if that's readily possible. Maybe the SQL Gateway's return type can be sent as part of the request so that the JobManager can send back results in an advantageous format? 8. Does ErrorType need to be marked as @PublicEvolving? I'm excited for the SQL client to support gateway mode! Given the change in design, do you think it'll still be part of the Flink 1.17 release? Cheers, Jim On Sun, Nov 27, 2022 at 8:54 PM Shengkai Fang <fskm...@gmail.com> wrote: > Hi, Jim and Alexey. > > We have written the proposal[1]. It would be appreciated if you can give us > some feedback. > > Best, > Shengkai > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-275%3A+Support+Remote+SQL+Client+Based+on+SQL+Gateway > > yu zelin <yuzelin....@gmail.com> 于2022年11月24日周四 12:06写道： > > > Hi Jim, > > Sorry for incorrect message in last reply. > > > Shengkai will help to your PR > > I mean Shengkai will help to review the PRs. And I will add some of your > > suggestion to my design. > > > > I think the POC code will be cherry-picked to my new design of SQL > Client. > > > 2022年11月23日 12:18，yu zelin <yuzelin....@gmail.com> 写道： > > > > > > Hi Jim, > > > Sorry for late response. Just another busy week :) > > > Last week, I’ve discussed within my team about my design. My teammates > > think it’s better to unify the local and remote mode, so I’ve > investigated > > and redesigned a new plan. I’ll inform you after the rewriting of FLIP > > finished (will be soon) and Shengkai will help to your PR. > > > > > > Best, > > > > > > Yu Zelin > > > > > >> 2022年11月22日 02:59，Jim Hughes <jhug...@confluent.io.INVALID> 写道： > > >> > > >> Hi Yu, Shengkai, > > >> > > >> As a quick update, I've had a chance to try out Yu's POC and it is > > working > > >> for me. (Admittedly, I haven't tried it too extensively; I only tried > > >> basic operations.) > > >> > > >> From my experiments, I did leave a few comments on > > >> https://github.com/apache/flink/pull/20958. > > >> > > >> Overall, the PRs I see look pretty good. Are they going to be merged > > >> soon? Anything else I can do to help? > > >> > > >> Cheers, > > >> > > >> Jim > > > > > > > >
