Hi, Shengkai

Thanks for initializing this FLIP,  supports application mode for SQL
Gateway is a great job. The FLIP design looks good to me.


I've read the FLIP-316 which mentions supporting deploying SQL job to
application clusters for interactive or non-interactive gateway mode.
But I noticed that you say this FLIP focuses on supporting deploy sql
scripts to the application cluster, does it mean that it only supports
non-interactive gateway mode?


Best,
Ron

Shengkai Fang <fskm...@gmail.com> 于2024年10月29日周二 14:46写道:

> Hi, HongShun. Thanks a lot for your response!
>
> > I wonder what is the scope of this FLIP, only aim for k8s, not including
> yarn?
>
> This FLIP also works for the yarn-application mode. But the yarn deployment
> doesn't support to ship the artifacts into the remote side. Please
> correct me if I'm wrong.
>
> > When talking about "custom", you mean these also will have some builtin
> implementations? If it exists, how to get their location in dfs based on
> SQL? Depending on some configuration or just convention over configuration.
>
> I think the builtin artfacts are catalogs/connectors/udf that are located
> at the $FLINK_HOME/lib directory.
>
> > Is the FLIP-316 still in need later?
>
> Yes. I think FLIP-316 is a great idea to use json plan to run the SQL Job
> and it brings great convenience to users to submit job in application mode
> in interactive mode.
>
> Best,
> Shengkai
>
>
>
>
> Shengkai Fang <fskm...@gmail.com> 于2024年10月29日周二 14:25写道:
>
> > Hi, Feng.
> >
> > Thanks for your response.
> >
> > > Will FLIP-316 merge into Flink 2.0 too ?
> >
> > I don't have time to finish the FLIP-316. So it depends on whether anyone
> > else can help to continue the discussion.
> >
> > > Will SqlDriver use the same one?
> >
> > Yes. We should reuse the same driver. I think the driver is the
> entrypoint
> > for the SQL script.
> >
> >
> > > The details SQL-client deploy SQL File to Cluster may not be very
> clear ?
> >
> > I have pushed a PoC branch about the change. Please take a look at
> > https://github.com/fsk119/flink/tree/application-mode (I don't test it
> > yet). At the mean time, I add a new method in the SqlGatewayService to
> > describe the change.
> >
> > Best,
> > Shengkai
> >
> >
> >
> > Feng Jin <jinfeng1...@gmail.com> 于2024年10月25日周五 21:15写道:
> >
> >> Hi, Shenkai
> >>
> >> Thank you for initiating this FLIP, I understand that supporting
> >> application mode for SQL gateway is very important. There are two small
> >> issues.
> >>
> >> > FLIP-480 is different from FLIP-316
> >>
> >>
> >>    1. Will FLIP-316 merge into Flink 2.0 too ?
> >>
> >>
> >>    2. Will SqlDriver use the same one?
> >>
> >>
> >> The details SQL-client deploy SQL File to Cluster may not be very clear
> ?
> >>
> >> I guess that some modifications need to be made to the client here,
> >> when deploying scripts in application mode, we need to call the newly
> >> added
> >> interface of the gateway service.
> >>
> >>
> >> Best,
> >> Feng
> >>
> >>
> >> On Thu, Oct 24, 2024 at 4:27 PM Shengkai Fang <fskm...@gmail.com>
> wrote:
> >>
> >> > Hi, everyone.
> >> >
> >> > I'd like to initiate a discussion about FLIP-480: Support to deploy
> >> script
> >> > in application mode[1].
> >> >
> >> > FLIP-480 supports to solve the problem that table program can not run
> in
> >> > application mode. Comparing to FLIP-316[2], FLIP-480 tries to compile
> >> the
> >> > script in the JM side, which is free from the limitation of the JSON
> >> > plan(JSON plan only serialize the identifier for temporary object) .
> >> >
> >> > For more details, please refer to the FLIP[1]. Welcome any feedback
> and
> >> > suggestions for improvement.
> >> >
> >> > Best,
> >> > Shengkai
> >> >
> >> > [1]
> >> >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-480%3A+Support+to+deploy+SQL+script+in+application+mode
> >> > [2]
> >> >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-316%3A+Support+application+mode+for+SQL+Gateway?src=contextnavpagetreemode
> >> >
> >>
> >
>

Reply via email to