Hi Shengkai, thanks for your reply. > REST API is the user interface. The REST API transforms the request to the invocation of the SQLGatewayService that is the one doing the work. We split the Gateway into the SQLGatewayService and Endpoint(REST API) and its benefit is that all the Endpoints share the same SQLGatewayService.
Finally, I got the point of SQLGatewayService, It is just for `Pluggable Endpoint Discovery`. I suggest you reorganize the FLIP, what Pluggable Endpoint Discovery is, and how users to add new Endpoint, before introducing SQLGatewayService. Then I have some doubts about the name SQLGatewayService, I think Service is seriously misleading, it is just an implementation class and not a Service. What about just `SQLGateway`? Best, Jingsong On Sat, May 7, 2022 at 4:03 PM Shengkai Fang <fskm...@gmail.com> wrote: > > Hi Martijn. > > It seems we have reached consensus to support the Gateway inside the Flink > code base. > > Hi, Echo. > > Thanks for your interest. > > > whether flink-sql-gateway should be retained in the Flink project. > > I think the discussion above is clear. It is the essential tool to provide > out-of-box experience for users. > > > For stream processing, what's the point of getting the result? Is it just > for debugging and how to unify with batch processing > > At the client side, when the OperationStaus is FINISHED, the client is able > to fetch the results from the Gateway. It is unified with the batch > processing now. > > > For batch processing, does the gateway need to cache all fetch results? > > No. In the Gateway, we will only buffer partial data and wait for the > client to consume. If the client takes away the buffered data, the Gateway > will clear the buffer and notify the fetcher thread starts to work until > the buffer is full again. The mechanism behind is much like the > producer-consumer model[1]. > > [1] https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem > > > Whether executing query and fetch results should be synchronous or > asynchronous? > > Do you mean the executeStatement response should also contain the result? I > don't think using the asynchronous API will cause performance regression. > In most cases, the network latency is about 100ms or lower. You can ping > www.baidu.com or www.google.com to test the latency. > > > When executing a query in flink-sql-client, I often find error logs of > FlinkJobNotFoundException. Should this be optimized? > > It's related to the current client implementation. I think you can open a > jira ticket, add more details and we can discuss the problem in the ticket. > In the FLIP-91, the Gateway can store the log per Operation. It may solve > your problems. > > > Best, > Shengkai
