Hi, zelin. Thanks for your update. LGTM.
Best, Shengkai yu zelin <yuzelin....@gmail.com> 于2022年12月20日周二 11:00写道： > Hi all, > > Recently I have received some feedbacks about the REST Endpoint > modification. The main point > is use ‘ResultSet’ as a part of FetchResultsResponseBody’ is not > convenient for serialization and > deserialization. So I think it’s better to introduce a new ‘ResultInfo’ to > carry the data. The ‘ResultInfo’ > will carry the row format information and be serialized and deserialized > according to the row format. > > In FLIP, I have modified the Section: Public Interface -> REST Endpoint > Modification. The main > change is the ‘FetchJsonFormatResultsResponseBody' and > ‘FetchPlainTextResultsResponseBody’ > was deleted and the ‘overview of fetching results REST API’ was minor > modified. > > Best, > Yu Zelin > > > 2022年12月13日 17:13，yu zelin <yuzelin....@gmail.com> 写道： > > > > Hi everyone, > > > > Sorry for the incorrect message in my last email. I want to start the > vote on Wednesday > > as long as there are no questions in this period. > > > > Best, > > Yu Zelin > > > > On Tue, Dec 13, 2022 at 5:08 PM yu zelin <yuzelin....@gmail.com <mailto: > yuzelin....@gmail.com>> wrote: > >> Hi, everyone, > >> > >> Looks like our new design is similar to Timo’s suggestion, and > considering that there has > >> no response from other devs for a long time, I want to start the vote > on Thursday. > >> > >> > >> Best, > >> Yu Zelin > >> > >>> 2022年12月13日 16:23，yu zelin <yuzelin....@gmail.com <mailto: > yuzelin....@gmail.com>> 写道： > >>> > >>> Hi, Timo, > >>> > >>> Thanks for your suggestion. Recently I have discussed with @Godfrey > He, @Shengkai Fang > >>> and @Jark Wu about the `RowFormat` (Thanks for all your suggestions). > We finally came to > >>> a consensus which is similar to your suggestion. The details are as > follows: > >>> > >>> 1. Add a REST query parameter ‘RowFormat’ = JSON/PLAIN_TEXT to tell > the REST Endpoint > >>> how to deserialize the RowData int ResultSet. > >>> > >>> JSON format means the RowData will be serialized to JSON format, > which contains original > >>> LogicalType information, so it can be deserialized back to RowData. > >>> > >>> PLAIN_TEXT format means the RowData will be serialized to > SQL-compliant, plain strings. > >>> The SQL Client can print the strings directly. > >>> > >>> The example URI for fetching results is: > >>> > > /v2/sessions/:session_handle/operations/:operation_handle/result/:token?rowFormat=PLAIN_TEXT > >>> > >>> 2. Introduce two response bodies for fetching results in two formats. > >>> > >>> For more details, please take a look at the FLIP [ > https://cwiki.apache.org/confluence/x/T48ODg]. > >>> I have updated it with an example of query response bodies in two > format in section: > >>> Public Interface -> REST Endpoint Modification. > >>> > >>>> 2022年12月12日 18:09，Timo Walther <twal...@apache.org <mailto: > twal...@apache.org>> 写道： > >>>> > >>>> Hi everyone, > >>>> > >>>> sorry to jump into this discussion so late. > >>>> > >>>> > So we decided to revert the RowFormat related changes and let the > client to resolve the print format. > >>>> > >>>> Could you elaborate a bit on this topic in the FLIP? I still believe > that we need 2 types of output formats. > >>>> > >>>> Format A: for the SQL Client CLI and other interactive notebooks that > just uses SQL CAST(... AS STRING) semantics executed on the server side > >>>> > >>>> Format B: for JDBC SDK or other machine-readable downstream libraries > >>>> > >>>> Take a TIMESTAMP WITH LOCAL TIME ZONE as an example. The string > representation depends on a session configuration option. Clients might not > be aware of this session option, so the formatting must happen on the > server side. > >>>> > >>>> However, when the downstream consumer is a library, maybe the library > would like to get the raw millis/nanos since epoch. > >>>> > >>>> Also nested rows and collections might be better encoded with format > B for libraries but interactive sessions are happy if nested types are > already formatted server-side, so not every client needs custom code for > the formatting. > >>>> > >>>> Regards, > >>>> Timo > >>>> > >>>> > >>>> > >>>> On 06.12.22 15:13, godfrey he wrote: > >>>>> Hi, zeklin > >>>>>> The CLI will use default print style for the non-query result. > >>>>> Please make sure the print results of EXPLAIN/DESC/SHOW CREATE TABLE > >>>>> commands are clear. > >>>>>> We think it’s better to add the root cause to the ErrorResponseBody. > >>>>> LGTM > >>>>> Best, > >>>>> Godfrey > >>>>> yu zelin <yuzelin....@gmail.com <mailto:yuzelin....@gmail.com>> > 于2022年12月6日周二 17:51写道： > >>>>>> > >>>>>> Hi, Godfrey > >>>>>> > >>>>>> Thanks for your feedback. Below is my thoughts about your questions. > >>>>>> > >>>>>> 1. About RowFormat. > >>>>>> I agree to your opinion. So we decided to revert the RowFormat > related changes > >>>>>> and let the client to resolve the print format. > >>>>>> > >>>>>> 2. About ContentType > >>>>>> I agree that the definition of the ContentType is not clear. But > how to define the > >>>>>> statement type is another big question. So, we decided to only tell > the query result > >>>>>> and non-query result apart. The CLI will use default print style > for the non-query > >>>>>> result. > >>>>>> > >>>>>> 3. About ErrorHandling > >>>>>> I think reuse the current ErrorResponseBody is good, but parse the > root cause > >>>>>> from the exception stack strings is quite hacking. We think it’s > better to add the > >>>>>> root cause to the ErrorResponseBody. > >>>>>> > >>>>>> 4. About Runtime REST API Modifications > >>>>>> I agree, too. This part is moved to the ‘Future Work’. > >>>>>> > >>>>>> Best, > >>>>>> Yu Zelin > >>>>>> > >>>>>> > >>>>>>> 2022年12月5日 18:33，godfrey he <godfre...@gmail.com <mailto: > godfre...@gmail.com>> 写道： > >>>>>>> > >>>>>>> Hi Zelin, > >>>>>>> > >>>>>>> Thanks for driving this discussion. > >>>>>>> > >>>>>>> I have a few comments, > >>>>>>> > >>>>>>>> Add RowFormat to ResultSet to indicate the format of rows. > >>>>>>> We should not require SqlGateway server to meet the display > >>>>>>> requirements of a CliClient. > >>>>>>> Because different CliClients may have different display style. The > >>>>>>> server just need to response the data, > >>>>>>> and the CliClient prints the result as needed. So RowFormat is not > needed. > >>>>>>> > >>>>>>>> Add ContentType to ResultSet to indicate what kind of data the > result contains. > >>>>>>> from my first sight, the values of ContentType are intersected, > such > >>>>>>> as: A select query will return QUERY_RESULT, > >>>>>>> but it also has JOB_ID. OTHER is too ambiguous, I don't know which > >>>>>>> kind of query will return OTHER. > >>>>>>> I recommend returning the concrete type for each statement, such as > >>>>>>> "CREATE TABLE" for "create table xx (...) with ()", > >>>>>>> "SELECT" for "select * from xxx". The statement type can be > maintained > >>>>>>> in `Operation`s. > >>>>>>> > >>>>>>>> Error Handling > >>>>>>> I think current design of error handling mechanism can meet the > >>>>>>> requirement of CliClient, we can get the root cause from > >>>>>>> the stack (see ErrorResponseBody#errors). If it becomes a common > >>>>>>> requirement (for many clients) in the future, > >>>>>>> we can introduce this interface. > >>>>>>> > >>>>>>>> Runtime REST API Modification for Local Client Migration > >>>>>>> I think this part is over-engineered, this part belongs to > optimization. > >>>>>>> The client does not require very high performance, the current > design > >>>>>>> can already meet our needs. > >>>>>>> If we find performance problems in the future, do such > optimizations. > >>>>>>> > >>>>>>> Best, > >>>>>>> Godfrey > >>>>>>> > >>>>>>> yu zelin <yuzelin....@gmail.com <mailto:yuzelin....@gmail.com>> > 于2022年12月5日周一 11:11写道： > >>>>>>>> > >>>>>>>> Hi, Shammon > >>>>>>>> > >>>>>>>> Thanks for your feedback. I think it’s good to support jdbc-sdk. > However, > >>>>>>>> it's not supported in the gateway side yet. In my opinion, this > FLIP is more > >>>>>>>> concerned with the SQL Client. How about put “supporting > jdbc-sdk” in > >>>>>>>> ‘Future Work’? We can discuss how to implement it in another > thread. > >>>>>>>> > >>>>>>>> Best, > >>>>>>>> Yu Zelin > >>>>>>>>> 2022年12月2日 18:12，Shammon FY <zjur...@gmail.com <mailto: > zjur...@gmail.com>> 写道： > >>>>>>>>> > >>>>>>>>> Hi zelin > >>>>>>>>> > >>>>>>>>> Thanks for driving this discussion. > >>>>>>>>> > >>>>>>>>> I notice that the sql-client will interact with sql-gateway by > `REST > >>>>>>>>> Client` in the `Executor` in the FLIP, how about introducing > jdbc-sdk for > >>>>>>>>> sql-gateway? > >>>>>>>>> > >>>>>>>>> Then the sql-client can connect the gateway with jdbc-sdk, on > the other > >>>>>>>>> hand, the other applications and tools such as jmeter can use > the jdbc-sdk > >>>>>>>>> to connect sql-gateway too. > >>>>>>>>> > >>>>>>>>> Best, > >>>>>>>>> Shammon > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Fri, Dec 2, 2022 at 4:10 PM yu zelin <yuzelin....@gmail.com > <mailto:yuzelin....@gmail.com>> wrote: > >>>>>>>>> > >>>>>>>>>> Hi Jim, > >>>>>>>>>> > >>>>>>>>>> Thanks for your feedback! > >>>>>>>>>> > >>>>>>>>>>> Should this configuration be mentioned in the FLIP? > >>>>>>>>>> > >>>>>>>>>> Sure. > >>>>>>>>>> > >>>>>>>>>>> some way for the server to be able to limit the number of > requests it > >>>>>>>>>> receives. > >>>>>>>>>> I’m sorry that this FLIP is dedicated in implementing the > Remote mode, so > >>>>>>>>>> we > >>>>>>>>>> didn't consider much about this. I think the option is enough > currently. > >>>>>>>>>> I will add > >>>>>>>>>> the improvement suggestions to the ‘Future Work’. > >>>>>>>>>> > >>>>>>>>>>> I wonder if two other options are possible > >>>>>>>>>> > >>>>>>>>>> To forward the raw format to gateway and then to client is > possible. The > >>>>>>>>>> raw > >>>>>>>>>> results from sink is in ‘CollectResultIterator#bufferedResult’. > First, we > >>>>>>>>>> can find > >>>>>>>>>> a way to get this result without wrapping it. Second, > constructing a > >>>>>>>>>> ‘InternalTypeInfo’. > >>>>>>>>>> We can construct it using the schema information (data’s > logical type). > >>>>>>>>>> After > >>>>>>>>>> construction, we can get the ’TypeSerializer’ to deserialize > the raw > >>>>>>>>>> result. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> 2022年12月1日 04:54，Jim Hughes <jhug...@confluent.io.INVALID> 写道： > >>>>>>>>>>> > >>>>>>>>>>> Hi Yu, > >>>>>>>>>>> > >>>>>>>>>>> Thanks for moving my comments to this thread! Also, thank you > for > >>>>>>>>>>> answering my questions; it is helping me understand the SQL > Gateway > >>>>>>>>>>> better. > >>>>>>>>>>> > >>>>>>>>>>> 5. > >>>>>>>>>>>> Our idea is to introduce a new session option (like > >>>>>>>>>>> 'sql-client.result.fetch-interval') to control > >>>>>>>>>>> the fetching requests sending frequency. What do you think? > >>>>>>>>>>> > >>>>>>>>>>> Should this configuration be mentioned in the FLIP? > >>>>>>>>>>> > >>>>>>>>>>> One slight concern I have with having > 'sql-client.result.fetch-interval' > >>>>>>>>>> as > >>>>>>>>>>> a session configuration is that users could set it low and > cause the > >>>>>>>>>> client > >>>>>>>>>>> to send a large volume of requests to the SQL gateway. > >>>>>>>>>>> > >>>>>>>>>>> Generally, I'd like to see some way for the server to be able > to limit > >>>>>>>>>> the > >>>>>>>>>>> number of requests it receives. If that really needs to be > done by a > >>>>>>>>>> proxy > >>>>>>>>>>> in front of the SQL gateway, that is fine as well. (To be > clear, I don't > >>>>>>>>>>> think my concern here should be blocking in any way.) > >>>>>>>>>>> > >>>>>>>>>>> 7. > >>>>>>>>>>>> What is the serialization lifecycle for results? > >>>>>>>>>>> > >>>>>>>>>>> I wonder if two other options are possible: > >>>>>>>>>>> 3) Could the Gateway just forward the result byte array? (Or > does the > >>>>>>>>>>> Gateway need to deserialize the response in order to > understand it for > >>>>>>>>>> some > >>>>>>>>>>> reason?) > >>>>>>>>>>> 4) Could the JobManager prepare the results in JSON? (Or > similarly could > >>>>>>>>>>> the Client read the format which the JobManager sends?) > >>>>>>>>>>> > >>>>>>>>>>> Thanks again! > >>>>>>>>>>> > >>>>>>>>>>> Cheers, > >>>>>>>>>>> > >>>>>>>>>>> Jim > >>>>>>>>>>> > >>>>>>>>>>> On Wed, Nov 30, 2022 at 9:40 AM yu zelin < > yuzelin....@gmail.com <mailto:yuzelin....@gmail.com>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Hi, all > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks Jim’s questions below. Here I’d like to reply to them. > >>>>>>>>>>>> > >>>>>>>>>>>>> 1. For the Client Parser, is it going to work with the > extended syntax > >>>>>>>>>>>>> from the Flink Table Store? > >>>>>>>>>>>>> > >>>>>>>>>>>>> 2. Relatedly, what will happen if an older Client tries to > handle > >>>>>>>>>>>> syntax > >>>>>>>>>>>>> that a newer service supports? (Suppose I use a 1.17 client > with a > >>>>>>>>>>>> 1.18 > >>>>>>>>>>>>> Gateway/system which has a new keyword. Is there anything > we should > >>>>>>>>>> be > >>>>>>>>>>>>> designing for upfront?) > >>>>>>>>>>>>> > >>>>>>>>>>>>> 3. How will client and server version mismatches be > handled? Will a > >>>>>>>>>>>>> single gateway be able to support multiple endpoint versions? > >>>>>>>>>>>>> 4. How are commands which change a session handled? Are > those sent > >>>>>>>>>> via > >>>>>>>>>>>>> an ExecuteStatementRequest? > >>>>>>>>>>>>> > >>>>>>>>>>>>> 5. The remote POC uses polling for getting back status and > getting > >>>>>>>>>> back > >>>>>>>>>>>>> results. Would it be possible to switch to web sockets or > some other > >>>>>>>>>>>>> mechanism to avoid polling? If polling is used for both, > the polling > >>>>>>>>>>>>> frequency should be different between local and remote > configurations. > >>>>>>>>>>>>> > >>>>>>>>>>>>> 6. What does this sentence mean? "The reason why we didn't > get the > >>>>>>>>>> sql > >>>>>>>>>>>>> type in client side is because it's hard for the lightweight > >>>>>>>>>>>> client-level > >>>>>>>>>>>>> parser to recognize some sql type sql, such as query with > CTE. " > >>>>>>>>>>>>> > >>>>>>>>>>>>> 7. What is the serialization lifecycle for results? It > makes sense to > >>>>>>>>>>>>> have some control over whether the gateway returns results > as SQL or > >>>>>>>>>>>> JSON. > >>>>>>>>>>>>> I'd love to see a way to avoid needing to serialize and > deserialize > >>>>>>>>>>>> results > >>>>>>>>>>>>> on the SQL Gateway if possible. I'm still new enough to the > project > >>>>>>>>>>>> that > >>>>>>>>>>>>> I'm not sure if that's readily possible. Maybe the SQL > Gateway's > >>>>>>>>>>>> return > >>>>>>>>>>>>> type can be sent as part of the request so that the > JobManager can > >>>>>>>>>> send > >>>>>>>>>>>>> back results in an advantageous format? > >>>>>>>>>>>>> > >>>>>>>>>>>>> 8. Does ErrorType need to be marked as @PublicEvolving? > >>>>>>>>>>>>> > >>>>>>>>>>>>> I'm excited for the SQL client to support gateway mode! > Given the > >>>>>>>>>> change > >>>>>>>>>>>>> in design, do you think it'll still be part of the Flink > 1.17 release? > >>>>>>>>>>>> > >>>>>>>>>>>> 1. ClientParser can work with new (and unknown) SQL syntax. > It is > >>>>>>>>>> because > >>>>>>>>>>>> if the > >>>>>>>>>>>> sql type is not recognized, the sql will be submitted to the > gateway > >>>>>>>>>>>> directly. > >>>>>>>>>>>> > >>>>>>>>>>>> For more information: Actually, the proposed ClientParser > only do two > >>>>>>>>>>>> things: > >>>>>>>>>>>> (1) Tell client commands (help, clear, etc) and sqls apart. > >>>>>>>>>>>> (2) parses several sql types (e.g. SHOW CREATE statement, we > can print > >>>>>>>>>> raw > >>>>>>>>>>>> string > >>>>>>>>>>>> for the SHOW CREATE result instead of table). Here the > recognization of > >>>>>>>>>>>> sql types > >>>>>>>>>>>> mostly affects the print style, and unrecognized sql also can > be > >>>>>>>>>> submitted > >>>>>>>>>>>> to cluster. > >>>>>>>>>>>> So the Client with new ClientParser can work compatible with > new syntax. > >>>>>>>>>>>> > >>>>>>>>>>>> 2. First, I'd like to explain that the gateway APIs and > supported syntax > >>>>>>>>>>>> is two things. > >>>>>>>>>>>> For example, ‘configureSession' and 'completeStatement' are > APIs. As > >>>>>>>>>>>> mentioned > >>>>>>>>>>>> in #1, the sql statements which syntax is unknown will be > submitted to > >>>>>>>>>> the > >>>>>>>>>>>> gateway, > >>>>>>>>>>>> and whether they can be executed normally depends on whether > the > >>>>>>>>>> execution > >>>>>>>>>>>> environment supports the syntax. > >>>>>>>>>>>> > >>>>>>>>>>>>> Is there anything we should be designing for upfront? > >>>>>>>>>>>> > >>>>>>>>>>>> The 'SqlGatewayRestAPIVersion’ has been introduced. But it is > for sql > >>>>>>>>>>>> gateway APIs. > >>>>>>>>>>>> > >>>>>>>>>>>> 3. > >>>>>>>>>>>>> How will client and server version mismatches be handled? > >>>>>>>>>>>> > >>>>>>>>>>>> A lower version client can work compatible with a higher > version gateway > >>>>>>>>>>>> because the > >>>>>>>>>>>> old interfaces won’t be deleted. When a higher version client > connects > >>>>>>>>>> to > >>>>>>>>>>>> a lower version > >>>>>>>>>>>> gateway, the client should notify the users if they try to use > >>>>>>>>>> unsupported > >>>>>>>>>>>> features. For > >>>>>>>>>>>> example, the client start option ‘-i’ means using > initialization file > >>>>>>>>>> to > >>>>>>>>>>>> initialize the session. > >>>>>>>>>>>> We plan to use the gateway’s ‘configureSession’ to implement > it. But > >>>>>>>>>> this > >>>>>>>>>>>> API is not > >>>>>>>>>>>> implemented in 1.16 Gateway (SqlGatewayRestAPIVersion = V1), > so if the > >>>>>>>>>>>> user try to > >>>>>>>>>>>> use ‘-i’ option to start the client with the 1.16 gateway, > the client > >>>>>>>>>>>> should tell the user that > >>>>>>>>>>>> Can’t execute ‘-i’ option with gateway which version is lower > than V2. > >>>>>>>>>>>> > >>>>>>>>>>>>> Will a single gateway be able to support multiple endpoint > versions? > >>>>>>>>>>>> > >>>>>>>>>>>> Currently, the gateway only starts a highest version endpoint > and the > >>>>>>>>>>>> higher version endpoint > >>>>>>>>>>>> is compatible with the lower version endpoint’s protocol. > >>>>>>>>>>>> > >>>>>>>>>>>> 4. Yes. Mostly, we use ’SET’ and ‘RESET’ statements to change > the > >>>>>>>>>> session > >>>>>>>>>>>> configuration. > >>>>>>>>>>>> Notice: the client can’t change the session (I mean, close > current > >>>>>>>>>> session > >>>>>>>>>>>> and open another > >>>>>>>>>>>> one). I’m not sure if you have need to change the session > itself? > >>>>>>>>>>>> > >>>>>>>>>>>> 5. > >>>>>>>>>>>>> Would it be possible to switch to web sockets or some other > mechanism > >>>>>>>>>>>> to avoid polling? > >>>>>>>>>>>> > >>>>>>>>>>>> Your suggestion is very good, but this flip is for supporting > the remote > >>>>>>>>>>>> client. How about taking > >>>>>>>>>>>> it as a future work? > >>>>>>>>>>>> > >>>>>>>>>>>>> If polling is used for both, the polling frequency should be > different > >>>>>>>>>>>> between local and remote > >>>>>>>>>>>> configurations. > >>>>>>>>>>>> > >>>>>>>>>>>> Our idea is to introduce a new session option (like > >>>>>>>>>>>> 'sql-client.result.fetch-interval') to control > >>>>>>>>>>>> the fetching requests sending frequency. What do you think? > >>>>>>>>>>>> > >>>>>>>>>>>> For more information: we are inclined to keep the polling > behavior in > >>>>>>>>>> this > >>>>>>>>>>>> version. For streaming > >>>>>>>>>>>> query, fetching results synchronously may occupy resources of > the > >>>>>>>>>> gateway > >>>>>>>>>>>> in a long period. > >>>>>>>>>>>> For example, if the job doesn’t return results for a long > time because > >>>>>>>>>> the > >>>>>>>>>>>> window has not been > >>>>>>>>>>>> triggered, the synchronously fetching will keep occupying the > >>>>>>>>>> connection. > >>>>>>>>>>>> In asynchronous > >>>>>>>>>>>> situation, the gateway can return a NOT_READY_RESULT quickly > and release > >>>>>>>>>>>> the resources > >>>>>>>>>>>> for other clients to use. I think we can make some > improvements for the > >>>>>>>>>>>> whole flow path in the > >>>>>>>>>>>> future. > >>>>>>>>>>>> > >>>>>>>>>>>> 6. Sorry for that there is mistakes in this sentence. Let me > make it > >>>>>>>>>> clear. > >>>>>>>>>>>> > >>>>>>>>>>>> We proposed to add 'ContentType' to indicates the result is > for what > >>>>>>>>>> kind > >>>>>>>>>>>> of sql. In this sentence, > >>>>>>>>>>>> I want to explain why we add 'ContentType' since the > ClientParser can > >>>>>>>>>>>> recognize the sql type too. > >>>>>>>>>>>> It is because the proposed ClientParser can't recognize > complex syntax. > >>>>>>>>>>>> For example, it can’t > >>>>>>>>>>>> recognize query with CTE. So the result should carry content > type > >>>>>>>>>>>> information to help the client to > >>>>>>>>>>>> know the sql type. For example, the > 'ContentType.QUERY_RESULT' indicates > >>>>>>>>>>>> the result is for a > >>>>>>>>>>>> query statement. > >>>>>>>>>>>> > >>>>>>>>>>>> 7. > >>>>>>>>>>>>> What is the serialization lifecycle for results? > >>>>>>>>>>>> > >>>>>>>>>>>> 1) Sink to JobManager : RowData -> Byte[ ] (serialize) > >>>>>>>>>>>> 2) JobManager to Gateway : Byte[ ] -> RowData (deserialize) > >>>>>>>>>>>> 3) Gateway sending : RowData -> Byte[ ] > (serialized to JSON > >>>>>>>>>>>> format) > >>>>>>>>>>>> 4) Client receiving : Byte[ ] -> RowData > (deserialize) > >>>>>>>>>>>> > >>>>>>>>>>>>> Maybe the SQL Gateway's return type can be sent as part of > the request > >>>>>>>>>>>> so that the > >>>>>>>>>>>> JobManager can send back results in an advantageous format? > >>>>>>>>>>>> > >>>>>>>>>>>> Yes. I think it's an improvement for the Client and Gateway. > We have > >>>>>>>>>> some > >>>>>>>>>>>> ideas. For example, > >>>>>>>>>>>> > >>>>>>>>>>>> 1) We can move the Gateway into the JobManager and reduce the > Ser/De > >>>>>>>>>> costs > >>>>>>>>>>>> from JM to Gateway. > >>>>>>>>>>>> 2) Or the Gateway can collect the data from the sink function > directly > >>>>>>>>>>>> instead of JobManager. > >>>>>>>>>>>> > >>>>>>>>>>>> But I think we can leave this as a future work and discuss in > another > >>>>>>>>>>>> thread. > >>>>>>>>>>>> > >>>>>>>>>>>> 8. Yes. > >>>>>>>>>>>> > >>>>>>>>>>>>> Do you think it'll still be part of the Flink 1.17 release? > >>>>>>>>>>>> Yes. We will try our best to finish the work. > >>>>>>>>>>>> > >>>>>>>>>>>> Feel free to talk to me if I’m wrong or you have any other > questions. > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>>> 2022年11月25日 11:48，yu zelin <yuzelin....@gmail.com <mailto: > yuzelin....@gmail.com>> 写道： > >>>>>>>>>>>>> > >>>>>>>>>>>>> Hi, all > >>>>>>>>>>>>> > >>>>>>>>>>>>> I want to initiate a discussion on the FLIP-275: Support > Remote SQL > >>>>>>>>>>>> Client Based on SQL Gateway[1]. > >>>>>>>>>>>>> The motivation of this FLIP is that the current SQL Client > allows only > >>>>>>>>>>>> local connection which can not satisfy > >>>>>>>>>>>>> the common need of connecting to a remote cluster. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Since the FLIP-91[2] has introduced SQL Gateway, we proposed > to > >>>>>>>>>>>> implement the Remote SQL Client > >>>>>>>>>>>>> based on SQL Gateway. In our design, we proposed two main > changes: > >>>>>>>>>>>>> > >>>>>>>>>>>>> 1. New remote mode client which performs connection to the > remote > >>>>>>>>>>>> gateway through REST API. > >>>>>>>>>>>>> 2. Migration of the current local mode client. We proposed > to refactor > >>>>>>>>>>>> the local client based on SQL Gateway > >>>>>>>>>>>>> to unify the interface for two modes. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Looking forward to your suggestions. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Best, > >>>>>>>>>>>>> Yu Zelin > >>>>>>>>>>>>> > >>>>>>>>>>>>> [1] https://cwiki.apache.org/confluence/x/T48ODg > >>>>>>>>>>>>> [2] https://cwiki.apache.org/confluence/x/rIyMC > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>> > >>>> > >>> > >> > >
