> > As I think about it, most likely orthogonal to the current > discussion, but this idea seems to be applicable for the > submitted job JARs as well. The same check could be done for all > the submitted files and their deps. IDK if that was your original > thought Xintong or not? >
IIUC, you mean applying the strict mode for the programming APIs as well? I'm not entirely sure about this. The programming APIs are much less stable compared to CLI interfaces. There are APIs being deprecated in almost every release. My concern is that this may end up with users never turning the strict mode on due to too many breaks. I'm a bit leaning towards starting with only applying the strict mode to CLI interfaces, and see how it goes. But I'm also open to other opinions. Best, Xintong On Wed, Jul 17, 2024 at 7:59 PM Ferenc Csaky <ferenc.cs...@pm.me.invalid> wrote: > Hi Xintong, Muhammet, > > Thank you for your thoughts! > > I agree with Xintong regarding 1), and 2). > > About making users aware of these compatibility guarantees, > documentation and CLI print deprecation is what I already aimed > for when I was merging the "run" and "run-application" actions > functionality [1], but I missed the CLI page, which is a very good > point to include. I will address that in a follow-up change. > > I also like the idea of a strict, and compatibility mode. I guess > that should kick in when a "@Public" AND "@Deprecated" annotation > combination is present on the called CLI action. > > With something like this at hand, I think applying the same > deprecation process and annotations for CLI actions that we already > have would make sense. Regarding the strict/compatibility strategy > itself, the straightforward and easy way IMO: > > - Strict mode: In case a @Public entity gets @Deprecated, execution > breaks right away. > - Compatibility mode: This should allow execution until the code > exists. > > I am not sure if bringing in more complexity would be beneficial, > as it would also make it harder for the users to understand. > > As I think about it, most likely orthogonal to the current > discussion, but this idea seems to be applicable for the > submitted job JARs as well. The same check could be done for all > the submitted files and their deps. IDK if that was your original > thought Xintong or not? > > Best, > Ferenc > > [1] > https://github.com/apache/flink/commit/e56b54db40a2afac420d8d8952707c2644ba633a > > > > On Tuesday, 9 July 2024 at 06:02, Xintong Song <tonysong...@gmail.com> > wrote: > > > > > > > I think there are three questions to be anwsered, and here are my two > cents. > > > > 1) How do we define the compatibility guarantees for cli interfaces? > > > > I'd be +1 for reusing the existing @Public and @PublicEvolving > annotations, > > as suggested by Ferenc. Having multiple sets of rules in the same project > > may easily confuse users. > > > > 2) What deprecation process is required for cli interfaces? > > > > If we are reusing the existing annotations, I think we'd better to have > the > > same deprecation process as well, for the same reason not to confuse > users. > > > > 3) How do we make user aware of the compatibility guarantees and > > deprecations of cli interfaces? > > > > I think there are several things that we can do. > > - In addition to codes and JavaDocs, we should also display the > annotations > > (Public / PublicEvolving / Experimental Dprecated) in documentation [1] > and > > cli helps (things you see when execute `<FLINK_HOME>/bin/flink -h`). > > > > - Print a warning message if a deprecated command / argument is used, as > > suggested by Muhammet. > > - We can also provide a strict / compatible mode. E.g., we use strict > mode > > by default, which fails if any deprecated interface is used. In the error > > message, we hint user that he/she can manually enable the compatible > mode, > > which allows deprecated interfaces. This should draw users' attention to > > the deprecation of the interfaces, while not block the adoption of a new > > Flink version with breaking changes. There are probably more details to > be > > considered, e.g., should we fail immediately once an interface is > > deprecated in strict mode, how long do we support a deprecated interface > in > > compatible mode, how to properly suggest users about the compatible mode > > while not encourage them to always stay in that mode, etc. > > > > Best, > > > > Xintong > > > > > > [1] > > > https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/cli/ > > > > > > > > On Mon, Jul 8, 2024 at 5:54 PM Muhammet Orazov > > mor+fl...@morazow.com.invalid wrote: > > > > > Hey Ferenc, > > > > > > Yes correct. My thoughts were based on finding tradeoff between > > > following fully deprecation process and leaner one for CLIs. > > > > > > Since cli are not like APIs, I think users would be aware of > > > deprecation only when were remove the commands. That is they > > > try to run their jobs with upgrade and it fails with action > > > not available. > > > > > > So maybe we don't have to follow fully API `@PublicEvolving` > > > process for this. > > > > > > Another maybe user friendly approach would be to inform with > > > warning that the `run-application` cli action will be dropped, > > > and suggest new action and migration on the log message. > > > > > > Best, > > > Muhammet > > > > > > On 2024-07-04 20:17, Ferenc Csaky wrote: > > > > > > > Hi Muhammet, > > > > > > > > Thank you for your thoughts! > > > > > > > > > After two minor releases, and on next major version bump, > > > > > we could drop the `run-application` method as suggested > > > > > on discussion by Xintong. > > > > > > > > Here, you describe the deprecation/removal process of a public > > > > API by the definition we have in the project now. So if the same > > > > applies to a CLI action, why should we not enforce such behavior > > > > for those as well? > > > > > > > > If following the same process for CLIs make sense, we should also > > > > enforce the same compatibility guarantees IMO. > > > > > > > > Best, > > > > Ferenc > > > > > > > > On Friday, 28 June 2024 at 09:30, Muhammet Orazov > > > > mor+fl...@morazow.com.INVALID wrote: > > > > > > > > > Hey Ferenc, > > > > > > > > > > Thanks for starting the discussion! > > > > > > > > > > I agree that the CLI is user facing, but I think we don't > > > > > have to treat it as other public APIs. > > > > > > > > > > I'd propose to throw user friendly exception for > > > > > `run-application` with suggestion to use `run` case instead. > > > > > This would make users aware of the change and require them > > > > > to migrate their scripts. > > > > > > > > > > After two minor releases, and on next major version bump, > > > > > we could drop the `run-application` method as suggested > > > > > on discussion by Xintong. > > > > > > > > > > Best, > > > > > Muhammet > > > > > > > > > > On 2024-06-26 15:33, Ferenc Csaky wrote: > > > > > > > > > > > Hello devs, > > > > > > I would like to open a discussion about considerations regarding > how > > > > > > to > > > > > > deprecate CLI > > > > > > actions, and what compatibility guarantees should apply to such > cases. > > > > > > The topic came up in > > > > > > a previous discussion [1] about a current FLIP to merge the run > and > > > > > > run-application > > > > > > behavior [2]. > > > > > > > > > > > > According to Xintong's previous inputs, currently the Flink CLI, > or > > > > > > its > > > > > > actions are not handled > > > > > > as public APIs by the existing definition (@Public or > @PublicEvolving > > > > > > annotated). So > > > > > > legally it would be possible to change CLIs anytime. I agree with > > > > > > Xintong that CLI actions > > > > > > should be considered as public APIs, and as such, compatibility > > > > > > guarantees should be > > > > > > provided. > > > > > > > > > > > > CLI actions are defined as private constants in CliFrontend [3], > so > > > > > > IMO the existing rules > > > > > > are not perfectly applicable as is. Both @Public and > @PublicEvolving > > > > > > can be applied to > > > > > > fields (although for @Public that is only true as per the > javadoc, > > > > > > technically it can only be > > > > > > applied to classes), so I think that could be a good approach > that is > > > > > > achievable with minimal > > > > > > effort and changes. > > > > > > > > > > > > It would also be possible to define a different process, but IMO > the > > > > > > more lightweight and > > > > > > maintainable the process the better, so it is a good thing if it > is > > > > > > not > > > > > > necessary to bring in > > > > > > new entities and/or rules to check and enforce compatibility. > > > > > > > > > > > > WDYT? > > > > > > > > > > > > Best, > > > > > > Ferenc > > > > > > > > > > > > [1] > https://lists.apache.org/thread/0vc8v3t7fr6w9hmwf9zbjbyk5c3bcj50 > > > > > > [2] > > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=311626179 > > > > > > > > > [3] > > > > > > > https://github.com/apache/flink/blob/27287a105f6585e89795e2a6cbffa8254abb6e57/flink-clients/src/main/java/org/apache/flink/client/cli/CliFrontend.java#L98 >
