Hi, Jim, Yanfei, Thanks for your comments! Let me reply in the order of the messages.
> I'd prefer sticking to the SQL standard if possible. Would > it be possible / sensible to allow for each syntax, perhaps > managed by a config setting? Correct me if I am wrong, but AFAIK variables are not part of the ANSI SQL standard. The '@' prefix is used by some widely used DB mgr, e.g. MySQL. Regarding having multiple resolution syntax, it would be possible, if we agree it adds value. Personally I do not have a strong opinion on that. > I'm new to Flink SQL and I'm curious if these variables can be > calculated from statements or expression [1]? Good point! The proposed solution would lack this functionality. On our platform, we have a working solution of this that was sufficient to solve the main problem we had to carry SQL between environments without change. At this point, variable values can only be literals, and they are automatically escaped during resolution. Except if they are resolved as a DDL statement property value. But if the community agrees that it would be useful to have the ability of calculated variables I would happily spend some time on possible solutions that makes sense in Flink. WDYT? Best, Ferenc On Thursday, March 28th, 2024 at 03:58, Yanfei Lei <fredia...@gmail.com> wrote: > > > Hi Ferenc, > > Thanks for the proposal, using SQLvariables to exclude > environment-specific configuration from code sounds like a good idea. > > I'm new to Flink SQL and I'm curious if these variables can be > calculated from statements or expression [1]? In FLIP, it seems that > the values are in the form of StringLiteral. > > > [1] > https://docs.databricks.com/en/sql/language-manual/sql-ref-syntax-aux-set-variable.html > > Jim Hughes jhug...@confluent.io.invalid 于2024年3月28日周四 04:54写道： > > > Hi Ferenc, > > > > Looks like a good idea. > > > > I'd prefer sticking to the SQL standard if possible. Would it be possible > > / sensible to allow for each syntax, perhaps managed by a config setting? > > > > Cheers, > > > > Jim > > > > On Tue, Mar 26, 2024 at 6:59 AM Ferenc Csaky ferenc.cs...@pm.me.invalid > > wrote: > > > > > Hello devs, > > > > > > I would like to start a discussion about FLIP-XXX: Introduce Flink SQL > > > variables [1]. > > > > > > The main motivation behing this change is to be able to abstract Flink SQL > > > from > > > environment-specific configuration and provide a way to carry jobs between > > > environments (e.g. dev-stage-prod) without the need to make changes in the > > > code. > > > It can also be a way to decouple sensitive information from the job code, > > > or help > > > with redundant literals. > > > > > > The main decision regarding the proposed solution is to handle the > > > variable resolution > > > as early as possible on the given string statement, so the whole operation > > > is an easy and > > > lightweight string replace. But this approach introduces some limitations > > > as well: > > > > > > - The executed SQL will always be the unresolved, raw string, so in case > > > of secrets > > > a DESC operation would show them. > > > - Changing the value of a variable can break code that uses that variable. > > > > > > For more details, please check the FLIP [1]. There is also a stale Jira > > > about this [2]. > > > > > > Looking forward to any comments and opinions! > > > > > > Thanks, > > > Ferenc > > > > > > [1] > > > https://docs.google.com/document/d/1-eUz-PBCdqNggG_irDT0X7fdL61ysuHOaWnrkZHb5Hc/edit?usp=sharing > > > [2] https://issues.apache.org/jira/browse/FLINK-17377 > > > > > -- > Best, > Yanfei
