> > I would prefer not to rely on the Jira for marking when something is > supposed to be deleted. If `@Deprecated(since, planned_to_remove_on)` would > have two obligatory parameters, there would be no way to "forget" about > marking it and it would be also self documenting (I don't imagine users > using JIRA to check this kind of things). We can have Jira tickets for > those things for tracking purposes on the JIRA release board, but relying > only on JIRA tickets I think is just asking for inconsistencies.
Yes, I totally agree with you. The annotation should be the way to document it. I just saw the Jira issue proposal as a good way to track these things within the team for code that is not visited regularly. But as I said, that depends on how each team organizes itself. I don't see this as a process everyone needs to follow. I agree it would depend on the feature, hence different features might have > longer or shorter "unstable" timeframes. But I'm afraid if we won't start > thinking about fixing this timeframe, we would too often end up with > perpetually "unstable" APIs. I don't know where I would draw the line > exactly, but assuming we want to have stable APIs, if something is marked > `@PublicEvolving` or `@Experimental` for 3 years, IMO it should be switched > to `@Public` by default (or be moved out of the main repo?). We could use the timestamp for the @PublicEvolving and @Experimental as an indicator to revisit the annotation. Let's say, we revisit each annotation after a specific amount of time (e.g. after two releases) to discuss whether the API is still evolving or actually stable. If it's stable, we change the annotation to @Public. If it's not stable, we update the timestamp. Best, Matthias On Wed, Jan 20, 2021 at 2:01 PM Piotr Nowojski <pnowoj...@apache.org> wrote: > Hi, > > I would prefer not to rely on the Jira for marking when something is > supposed to be deleted. If `@Deprecated(since, planned_to_remove_on)` would > have two obligatory parameters, there would be no way to "forget" about > marking it and it would be also self documenting (I don't imagine users > using JIRA to check this kind of things). We can have Jira tickets for > those things for tracking purposes on the JIRA release board, but relying > only on JIRA tickets I think is just asking for inconsistencies. > > > Is it actually possible to have a fixed timeframe for these annotations > to change? > > I would imagine that it depends on the underlying feature how long an > API is @PublicEvolving or @Experimental? > > I agree it would depend on the feature, hence different features might > have longer or shorter "unstable" timeframes. But I'm afraid if we won't > start thinking about fixing this timeframe, we would too often end up with > perpetually "unstable" APIs. I don't know where I would draw the line > exactly, but assuming we want to have stable APIs, if something is marked > `@PublicEvolving` or `@Experimental` for 3 years, IMO it should be switched > to `@Public` by default (or be moved out of the main repo?). > > Piotrek > > śr., 20 sty 2021 o 09:54 Matthias Pohl <matth...@ververica.com> > napisał(a): > >> Thanks Timo for opening this discussion. >> >> +1 I like the idea of adding a deprecation deadline and/or information >> when >> the >> functionality was deprecated. It looks like this is already done in the >> PyFlink code. >> >> Creating a JIRA issue for removing the functionality, as Till suggested, >> might help to >> maintain this process of removing the deprecated functionality. I'd prefer >> that over >> relying on the release manager (assuming that he/she would run the check >> as >> part >> of the release process) to identify functionality that should have been >> removed as >> part of the release. But ok, that might be a team decision. >> >> For the connectors: Can't we assume that users would reach out to us if we >> deprecate >> a connector assuming that they can conclude that this connector will, >> otherwise, disappear. >> Maybe, that needs to be mentioned in the deprecation information as well, >> then. >> This would have the benefit of getting direct feedback about how much a >> connector is still in >> use and may open the doors for other contributors to offer help like it >> happened for the >> Mesos support [1]. >> >> And about the idea of adding such deadlines to @Public, @PublicEvolving, >> and @Experimental: >> Is it actually possible to have a fixed timeframe for these annotations to >> change? I would >> imagine that it depends on the underlying feature how long an API >> is @PublicEvolving or >> @Experimental? But it sounds still like a good idea to trigger warnings >> for >> those annotations >> in case they haven't been touched for a while. Therefore, I would second >> this suggestion. >> >> Best, >> Matthias >> >> [1] >> >> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/SURVEY-Remove-Mesos-support-td45974.html#a45985 >> >> On Tue, Jan 19, 2021 at 10:15 AM Till Rohrmann <trohrm...@apache.org> >> wrote: >> >> > Thanks a lot for starting this discussion Timo. I like the idea of >> setting >> > more explicit guidelines for deprecating functionality. >> > >> > I really like the idea of adding with the @Deprecated annotation since >> when >> > the function is deprecated. Based on that one can simply search for >> > features which should be removed in a given release. Alternatively, one >> > could as you said also state the removal version. >> > >> > I think what also works is to directly create a critical JIRA issue with >> > removing functionality as soon as one deprecates something. The problem >> was >> > often that after deprecating something, it gets forgotten. >> > >> > For dropping connectors I am a bit uncertain. From a project management >> > perspective it sounds like a good idea to not have to support connectors >> > which are no longer supported for some time. However, what if this >> > connector is still very popular and in heavy use by our users? Just >> because >> > an external system or a version of it is no longer maintained does not >> mean >> > that the system is no longer used. I think our current practice with >> trying >> > to judge whether our users still use this feature/connector works >> somewhat. >> > On the other hand, having these guidelines would probably make it >> easier to >> > argue to remove something even if there are still a couple of users. >> > >> > Cheers, >> > Till >> > >> > On Mon, Jan 18, 2021 at 11:37 AM Yun Gao <yungao...@aliyun.com.invalid> >> > wrote: >> > >> > > Hi, >> > > >> > > Very thanks for @Timo to initiate the discussion! >> > > >> > > I would also +1 for providing some informations to users via >> annotations >> > > or documents in advanced to not suprise users before we actually >> remove >> > > the legacy code. >> > > If we finally decide to change one functionality that user could >> sense, >> > > perhaps one >> > > premise is that Flink has provided a replacement for that one and >> users >> > > could transfer their >> > > applications easily. Then we might also consider have one dedicated >> > > documentation page >> > > to list the functionalities to change and how users could do the >> > transfer. >> > > >> > > To make the decision of whether to remove some legacy code, we might >> also >> > > consider to have a survey >> > > like the one we did for mesos support [1] to see how this >> functionality >> > is >> > > used. >> > > >> > > Best, >> > > Yun >> > > >> > > >> > > [1] >> > > >> > >> https://lists.apache.org/thread.html/r139b11190a6d1f09c9e44d5fa985fd8d310347e66d2324ec1f0c2d87%40%3Cuser.flink.apache.org%3E >> > > >> > > >> > > >> > > ------------------Original Mail ------------------ >> > > Sender:Piotr Nowojski <pnowoj...@apache.org> >> > > Send Date:Mon Jan 18 18:23:36 2021 >> > > Recipients:dev <dev@flink.apache.org> >> > > Subject:Re: [DISCUSS] Dealing with deprecated and legacy code in Flink >> > > Hi Timo, >> > > >> > > Thanks for starting this discussion. I'm not sure how we should >> approach >> > > this topic and what should be our final recommendation, but definitely >> > > clearing up a couple of things would be helpful. >> > > >> > > For starters, I agree it would be good to have some more information, >> > > besides just "@Deprecated" annotations. Is it possible to extend >> > > annotations with informations like: >> > > - from which version was it deprecated >> > > - when is it planned to be removed (we could always mark `2.0` as >> "never" >> > > ;) ) >> > > - add maybe some pre/post release step of verifying that removal has >> > > actually happened? >> > > >> > > ? >> > > >> > > On the other hand, I think it's very important to maintain backward >> > > compatibility with Flink as much as possible. As a developer I don't >> > > like dealing with this, but as a user I hate dealing with incompatible >> > > upgrades even more. So all in all, I would be in favour of putting >> more >> > > effort not in deprecating and removing APIs, but making sure that they >> > are >> > > stable. >> > > >> > > Stephan Ewan also raised a point sometime ago, that in the recent >> past, >> > we >> > > developed a habit of marking everything as `@Experimental` or >> > > `@PublicEvolving` and leaving it as that forever. Maybe we should also >> > > include deadlines (2 releases since introduction?) for changing >> > > `@Experimental`/`@PublicEvolving` into `@Public` in this kind of >> > > guidelines/automated checks? >> > > >> > > Piotrek >> > > >> > > pt., 15 sty 2021 o 13:56 Timo Walther <twal...@apache.org> >> napisał(a): >> > > >> > > > Hi everyone, >> > > > >> > > > I would like to start a discussion how we treat deprecated and >> legacy >> > > > code in Flink in the future. During the last years, our code base >> has >> > > > grown quite a bit and a couple of interfaces and components have >> been >> > > > reworked on the way. >> > > > >> > > > I'm sure each component has a few legacy parts that are waiting for >> > > > removal. Apart from keeping outdated API around for a couple of >> > releases >> > > > until users have updated their code, it is also often easier to just >> > put >> > > > a @Deprecation annotation and postpone the actual change. >> > > > >> > > > When looking at the current code, we have duplicated SQL planners, >> > > > duplicated APIs (DataSet/DataStream), duplicated source/sink >> > interfaces, >> > > > outdated connectors (Elasticsearch 5?) and dependencies (Scala >> 2.11?). >> > > > >> > > > I'm wondering whether we should come up with some legacy/deprecation >> > > > guidelines for the future. >> > > > >> > > > Some examples: >> > > > >> > > > - I could imagine new Flink-specific annotations for documenting (in >> > > > code) in which version an interface was deprecated and when the >> planned >> > > > removal should take place. >> > > > - Or guidelines that we drop a connector when the external project >> does >> > > > not maintain the version for 6 months etc. >> > > > >> > > > Plannable removal dates should also help users to not be surprised >> when >> > > > a connector or Scala version is not supported anymore. >> > > > >> > > > What do you think? I'm very happy to hear more opinions. >> > > > >> > > > Regards, >> > > > Timo >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > >> >
