Hi, everyone. Thanks for all the inputs. Hope my feedback can address most of questions.
After the long discussion, I think we have reached the consensus about the design. If the discussion doesn't get more response, I think we can start the vote tomorrow. Best, Shengkai Shengkai Fang <fskm...@gmail.com> 于2022年5月9日周一 14:18写道： > Hi JingSong. > > Thanks for your feedback. > > > reorganize the FLIP, what Pluggable Endpoint Discovery is, and how users > to add new Endpoint, before introducing SQLGatewayService. > > I update the FLIP: reorganize the order and add more details. > > > Then I have some doubts about the name SQLGatewayService, I think > > Service is seriously misleading, it is just an implementation class > > and not a Service. > > After discuss with Jingsong offline, we agree to still use the > SQLGatewayService. According to the defination of the wiki[1], service > refers to a software functionality or a set of software functionalities > with a purpose that different clients can reuse for different purposes. > Here the GatewayService plays the same role. > > Best, > Shengkai > > [1] https://en.wikipedia.org/wiki/Service_(systems_architecture) > > Jingsong Li <jingsongl...@gmail.com> 于2022年5月7日周六 16:55写道： > >> Hi Shengkai, thanks for your reply. >> >> > REST API is the user interface. The REST API transforms the request to >> the >> invocation of the SQLGatewayService that is the one doing the work. We >> split the Gateway into the SQLGatewayService and Endpoint(REST API) and >> its >> benefit is that all the Endpoints share the same SQLGatewayService. >> >> Finally, I got the point of SQLGatewayService, It is just for >> `Pluggable Endpoint Discovery`. >> I suggest you reorganize the FLIP, what Pluggable Endpoint Discovery >> is, and how users to add new Endpoint, before introducing >> SQLGatewayService. >> >> Then I have some doubts about the name SQLGatewayService, I think >> Service is seriously misleading, it is just an implementation class >> and not a Service. >> >> What about just `SQLGateway`? >> >> Best, >> Jingsong >> >> On Sat, May 7, 2022 at 4:03 PM Shengkai Fang <fskm...@gmail.com> wrote: >> > >> > Hi Martijn. >> > >> > It seems we have reached consensus to support the Gateway inside the >> Flink >> > code base. >> > >> > Hi, Echo. >> > >> > Thanks for your interest. >> > >> > > whether flink-sql-gateway should be retained in the Flink project. >> > >> > I think the discussion above is clear. It is the essential tool to >> provide >> > out-of-box experience for users. >> > >> > > For stream processing, what's the point of getting the result? Is it >> just >> > for debugging and how to unify with batch processing >> > >> > At the client side, when the OperationStaus is FINISHED, the client is >> able >> > to fetch the results from the Gateway. It is unified with the batch >> > processing now. >> > >> > > For batch processing, does the gateway need to cache all fetch >> results? >> > >> > No. In the Gateway, we will only buffer partial data and wait for the >> > client to consume. If the client takes away the buffered data, the >> Gateway >> > will clear the buffer and notify the fetcher thread starts to work until >> > the buffer is full again. The mechanism behind is much like the >> > producer-consumer model[1]. >> > >> > [1] https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem >> > >> > > Whether executing query and fetch results should be synchronous or >> > asynchronous? >> > >> > Do you mean the executeStatement response should also contain the >> result? I >> > don't think using the asynchronous API will cause performance >> regression. >> > In most cases, the network latency is about 100ms or lower. You can ping >> > www.baidu.com or www.google.com to test the latency. >> > >> > > When executing a query in flink-sql-client, I often find error logs of >> > FlinkJobNotFoundException. Should this be optimized? >> > >> > It's related to the current client implementation. I think you can open >> a >> > jira ticket, add more details and we can discuss the problem in the >> ticket. >> > In the FLIP-91, the Gateway can store the log per Operation. It may >> solve >> > your problems. >> > >> > >> > Best, >> > Shengkai >> >
