Hi, Shengkai Thanks for initializing this FLIP, supports application mode for SQL Gateway is a great job. The FLIP design looks good to me.
I've read the FLIP-316 which mentions supporting deploying SQL job to application clusters for interactive or non-interactive gateway mode. But I noticed that you say this FLIP focuses on supporting deploy sql scripts to the application cluster, does it mean that it only supports non-interactive gateway mode? Best, Ron Shengkai Fang <fskm...@gmail.com> 于2024年10月29日周二 14:46写道： > Hi, HongShun. Thanks a lot for your response! > > > I wonder what is the scope of this FLIP, only aim for k8s, not including > yarn? > > This FLIP also works for the yarn-application mode. But the yarn deployment > doesn't support to ship the artifacts into the remote side. Please > correct me if I'm wrong. > > > When talking about "custom", you mean these also will have some builtin > implementations? If it exists, how to get their location in dfs based on > SQL? Depending on some configuration or just convention over configuration. > > I think the builtin artfacts are catalogs/connectors/udf that are located > at the $FLINK_HOME/lib directory. > > > Is the FLIP-316 still in need later? > > Yes. I think FLIP-316 is a great idea to use json plan to run the SQL Job > and it brings great convenience to users to submit job in application mode > in interactive mode. > > Best, > Shengkai > > > > > Shengkai Fang <fskm...@gmail.com> 于2024年10月29日周二 14:25写道： > > > Hi, Feng. > > > > Thanks for your response. > > > > > Will FLIP-316 merge into Flink 2.0 too ? > > > > I don't have time to finish the FLIP-316. So it depends on whether anyone > > else can help to continue the discussion. > > > > > Will SqlDriver use the same one? > > > > Yes. We should reuse the same driver. I think the driver is the > entrypoint > > for the SQL script. > > > > > > > The details SQL-client deploy SQL File to Cluster may not be very > clear ? > > > > I have pushed a PoC branch about the change. Please take a look at > > https://github.com/fsk119/flink/tree/application-mode (I don't test it > > yet). At the mean time, I add a new method in the SqlGatewayService to > > describe the change. > > > > Best, > > Shengkai > > > > > > > > Feng Jin <jinfeng1...@gmail.com> 于2024年10月25日周五 21:15写道： > > > >> Hi, Shenkai > >> > >> Thank you for initiating this FLIP, I understand that supporting > >> application mode for SQL gateway is very important. There are two small > >> issues. > >> > >> > FLIP-480 is different from FLIP-316 > >> > >> > >> 1. Will FLIP-316 merge into Flink 2.0 too ? > >> > >> > >> 2. Will SqlDriver use the same one? > >> > >> > >> The details SQL-client deploy SQL File to Cluster may not be very clear > ? > >> > >> I guess that some modifications need to be made to the client here, > >> when deploying scripts in application mode, we need to call the newly > >> added > >> interface of the gateway service. > >> > >> > >> Best, > >> Feng > >> > >> > >> On Thu, Oct 24, 2024 at 4:27 PM Shengkai Fang <fskm...@gmail.com> > wrote: > >> > >> > Hi, everyone. > >> > > >> > I'd like to initiate a discussion about FLIP-480: Support to deploy > >> script > >> > in application mode[1]. > >> > > >> > FLIP-480 supports to solve the problem that table program can not run > in > >> > application mode. Comparing to FLIP-316[2], FLIP-480 tries to compile > >> the > >> > script in the JM side, which is free from the limitation of the JSON > >> > plan(JSON plan only serialize the identifier for temporary object) . > >> > > >> > For more details, please refer to the FLIP[1]. Welcome any feedback > and > >> > suggestions for improvement. > >> > > >> > Best, > >> > Shengkai > >> > > >> > [1] > >> > > >> > > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-480%3A+Support+to+deploy+SQL+script+in+application+mode > >> > [2] > >> > > >> > > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-316%3A+Support+application+mode+for+SQL+Gateway?src=contextnavpagetreemode > >> > > >> > > >
