I also have some questions: 1. Whether all SQL commands such as DDL & DML & SELECT are supported. 2. How to determine JobID and return JobID & ClusterId from the application cluster 3. How to dynamically download the JAR specified by the user when submitting the sql script, and whether it is possible to specify a local jar?
Best, Ron Ron Liu <ron9....@gmail.com> 于2024年10月30日周三 10:57写道： > Hi, Shengkai > > Thanks for initializing this FLIP, supports application mode for SQL > Gateway is a great job. The FLIP design looks good to me. > > > I've read the FLIP-316 which mentions supporting deploying SQL job to > application clusters for interactive or non-interactive gateway mode. > But I noticed that you say this FLIP focuses on supporting deploy sql > scripts to the application cluster, does it mean that it only supports > non-interactive gateway mode? > > > Best, > Ron > > Shengkai Fang <fskm...@gmail.com> 于2024年10月29日周二 14:46写道： > >> Hi, HongShun. Thanks a lot for your response! >> >> > I wonder what is the scope of this FLIP, only aim for k8s, not including >> yarn? >> >> This FLIP also works for the yarn-application mode. But the yarn >> deployment >> doesn't support to ship the artifacts into the remote side. Please >> correct me if I'm wrong. >> >> > When talking about "custom", you mean these also will have some builtin >> implementations? If it exists, how to get their location in dfs based on >> SQL? Depending on some configuration or just convention over >> configuration. >> >> I think the builtin artfacts are catalogs/connectors/udf that are located >> at the $FLINK_HOME/lib directory. >> >> > Is the FLIP-316 still in need later? >> >> Yes. I think FLIP-316 is a great idea to use json plan to run the SQL Job >> and it brings great convenience to users to submit job in application mode >> in interactive mode. >> >> Best, >> Shengkai >> >> >> >> >> Shengkai Fang <fskm...@gmail.com> 于2024年10月29日周二 14:25写道： >> >> > Hi, Feng. >> > >> > Thanks for your response. >> > >> > > Will FLIP-316 merge into Flink 2.0 too ? >> > >> > I don't have time to finish the FLIP-316. So it depends on whether >> anyone >> > else can help to continue the discussion. >> > >> > > Will SqlDriver use the same one? >> > >> > Yes. We should reuse the same driver. I think the driver is the >> entrypoint >> > for the SQL script. >> > >> > >> > > The details SQL-client deploy SQL File to Cluster may not be very >> clear ? >> > >> > I have pushed a PoC branch about the change. Please take a look at >> > https://github.com/fsk119/flink/tree/application-mode (I don't test it >> > yet). At the mean time, I add a new method in the SqlGatewayService to >> > describe the change. >> > >> > Best, >> > Shengkai >> > >> > >> > >> > Feng Jin <jinfeng1...@gmail.com> 于2024年10月25日周五 21:15写道： >> > >> >> Hi, Shenkai >> >> >> >> Thank you for initiating this FLIP, I understand that supporting >> >> application mode for SQL gateway is very important. There are two small >> >> issues. >> >> >> >> > FLIP-480 is different from FLIP-316 >> >> >> >> >> >> 1. Will FLIP-316 merge into Flink 2.0 too ? >> >> >> >> >> >> 2. Will SqlDriver use the same one? >> >> >> >> >> >> The details SQL-client deploy SQL File to Cluster may not be very >> clear ? >> >> >> >> I guess that some modifications need to be made to the client here, >> >> when deploying scripts in application mode, we need to call the newly >> >> added >> >> interface of the gateway service. >> >> >> >> >> >> Best, >> >> Feng >> >> >> >> >> >> On Thu, Oct 24, 2024 at 4:27 PM Shengkai Fang <fskm...@gmail.com> >> wrote: >> >> >> >> > Hi, everyone. >> >> > >> >> > I'd like to initiate a discussion about FLIP-480: Support to deploy >> >> script >> >> > in application mode[1]. >> >> > >> >> > FLIP-480 supports to solve the problem that table program can not >> run in >> >> > application mode. Comparing to FLIP-316[2], FLIP-480 tries to compile >> >> the >> >> > script in the JM side, which is free from the limitation of the JSON >> >> > plan(JSON plan only serialize the identifier for temporary object) . >> >> > >> >> > For more details, please refer to the FLIP[1]. Welcome any feedback >> and >> >> > suggestions for improvement. >> >> > >> >> > Best, >> >> > Shengkai >> >> > >> >> > [1] >> >> > >> >> > >> >> >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-480%3A+Support+to+deploy+SQL+script+in+application+mode >> >> > [2] >> >> > >> >> > >> >> >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-316%3A+Support+application+mode+for+SQL+Gateway?src=contextnavpagetreemode >> >> > >> >> >> > >> >
