I would prefer not to rely on the Jira for marking when something is
supposed to be deleted. If `@Deprecated(since, planned_to_remove_on)`
would
have two obligatory parameters, there would be no way to "forget" about
marking it and it would be also self documenting (I don't imagine users
using JIRA to check this kind of things). We can have Jira tickets for
those things for tracking purposes on the JIRA release board, but relying
only on JIRA tickets I think is just asking for inconsistencies.
Yes, I totally agree with you. The annotation should be the way to document
it. I just saw the Jira issue proposal as a good way to track these things
within the team for code that is not visited regularly. But as I said, that
depends on how each team organizes itself. I don't see this as a process
everyone needs to follow.
I agree it would depend on the feature, hence different features might have
longer or shorter "unstable" timeframes. But I'm afraid if we won't start
thinking about fixing this timeframe, we would too often end up with
perpetually "unstable" APIs. I don't know where I would draw the line
exactly, but assuming we want to have stable APIs, if something is marked
`@PublicEvolving` or `@Experimental` for 3 years, IMO it should be
switched
to `@Public` by default (or be moved out of the main repo?).
We could use the timestamp for the @PublicEvolving and @Experimental as an
indicator to revisit the annotation. Let's say, we revisit each annotation
after a specific amount of time (e.g. after two releases) to discuss
whether the API is still evolving or actually stable. If it's stable, we
change the annotation to @Public. If it's not stable, we update the
timestamp.
Best,
Matthias
On Wed, Jan 20, 2021 at 2:01 PM Piotr Nowojski <pnowoj...@apache.org>
wrote:
Hi,
I would prefer not to rely on the Jira for marking when something is
supposed to be deleted. If `@Deprecated(since, planned_to_remove_on)`
would
have two obligatory parameters, there would be no way to "forget" about
marking it and it would be also self documenting (I don't imagine users
using JIRA to check this kind of things). We can have Jira tickets for
those things for tracking purposes on the JIRA release board, but relying
only on JIRA tickets I think is just asking for inconsistencies.
Is it actually possible to have a fixed timeframe for these annotations
to change?
I would imagine that it depends on the underlying feature how long an
API is @PublicEvolving or @Experimental?
I agree it would depend on the feature, hence different features might
have longer or shorter "unstable" timeframes. But I'm afraid if we won't
start thinking about fixing this timeframe, we would too often end up
with
perpetually "unstable" APIs. I don't know where I would draw the line
exactly, but assuming we want to have stable APIs, if something is marked
`@PublicEvolving` or `@Experimental` for 3 years, IMO it should be
switched
to `@Public` by default (or be moved out of the main repo?).
Piotrek
śr., 20 sty 2021 o 09:54 Matthias Pohl <matth...@ververica.com>
napisał(a):
Thanks Timo for opening this discussion.
+1 I like the idea of adding a deprecation deadline and/or information
when
the
functionality was deprecated. It looks like this is already done in the
PyFlink code.
Creating a JIRA issue for removing the functionality, as Till suggested,
might help to
maintain this process of removing the deprecated functionality. I'd
prefer
that over
relying on the release manager (assuming that he/she would run the check
as
part
of the release process) to identify functionality that should have been
removed as
part of the release. But ok, that might be a team decision.
For the connectors: Can't we assume that users would reach out to us if
we
deprecate
a connector assuming that they can conclude that this connector will,
otherwise, disappear.
Maybe, that needs to be mentioned in the deprecation information as
well,
then.
This would have the benefit of getting direct feedback about how much a
connector is still in
use and may open the doors for other contributors to offer help like it
happened for the
Mesos support [1].
And about the idea of adding such deadlines to @Public, @PublicEvolving,
and @Experimental:
Is it actually possible to have a fixed timeframe for these annotations
to
change? I would
imagine that it depends on the underlying feature how long an API
is @PublicEvolving or
@Experimental? But it sounds still like a good idea to trigger warnings
for
those annotations
in case they haven't been touched for a while. Therefore, I would second
this suggestion.
Best,
Matthias
[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/SURVEY-Remove-Mesos-support-td45974.html#a45985
On Tue, Jan 19, 2021 at 10:15 AM Till Rohrmann <trohrm...@apache.org>
wrote:
Thanks a lot for starting this discussion Timo. I like the idea of
setting
more explicit guidelines for deprecating functionality.
I really like the idea of adding with the @Deprecated annotation since
when
the function is deprecated. Based on that one can simply search for
features which should be removed in a given release. Alternatively,
one
could as you said also state the removal version.
I think what also works is to directly create a critical JIRA issue
with
removing functionality as soon as one deprecates something. The
problem
was
often that after deprecating something, it gets forgotten.
For dropping connectors I am a bit uncertain. From a project
management
perspective it sounds like a good idea to not have to support
connectors
which are no longer supported for some time. However, what if this
connector is still very popular and in heavy use by our users? Just
because
an external system or a version of it is no longer maintained does not
mean
that the system is no longer used. I think our current practice with
trying
to judge whether our users still use this feature/connector works
somewhat.
On the other hand, having these guidelines would probably make it
easier to
argue to remove something even if there are still a couple of users.
Cheers,
Till
On Mon, Jan 18, 2021 at 11:37 AM Yun Gao <yungao...@aliyun.com.invalid
wrote:
Hi,
Very thanks for @Timo to initiate the discussion!
I would also +1 for providing some informations to users via
annotations
or documents in advanced to not suprise users before we actually
remove
the legacy code.
If we finally decide to change one functionality that user could
sense,
perhaps one
premise is that Flink has provided a replacement for that one and
users
could transfer their
applications easily. Then we might also consider have one dedicated
documentation page
to list the functionalities to change and how users could do the
transfer.
To make the decision of whether to remove some legacy code, we might
also
consider to have a survey
like the one we did for mesos support [1] to see how this
functionality
is
used.
Best,
Yun
[1]
https://lists.apache.org/thread.html/r139b11190a6d1f09c9e44d5fa985fd8d310347e66d2324ec1f0c2d87%40%3Cuser.flink.apache.org%3E
------------------Original Mail ------------------
Sender:Piotr Nowojski <pnowoj...@apache.org>
Send Date:Mon Jan 18 18:23:36 2021
Recipients:dev <dev@flink.apache.org>
Subject:Re: [DISCUSS] Dealing with deprecated and legacy code in
Flink
Hi Timo,
Thanks for starting this discussion. I'm not sure how we should
approach
this topic and what should be our final recommendation, but
definitely
clearing up a couple of things would be helpful.
For starters, I agree it would be good to have some more
information,
besides just "@Deprecated" annotations. Is it possible to extend
annotations with informations like:
- from which version was it deprecated
- when is it planned to be removed (we could always mark `2.0` as
"never"
;) )
- add maybe some pre/post release step of verifying that removal has
actually happened?
?
On the other hand, I think it's very important to maintain backward
compatibility with Flink as much as possible. As a developer I don't
like dealing with this, but as a user I hate dealing with
incompatible
upgrades even more. So all in all, I would be in favour of putting
more
effort not in deprecating and removing APIs, but making sure that
they
are
stable.
Stephan Ewan also raised a point sometime ago, that in the recent
past,
we
developed a habit of marking everything as `@Experimental` or
`@PublicEvolving` and leaving it as that forever. Maybe we should
also
include deadlines (2 releases since introduction?) for changing
`@Experimental`/`@PublicEvolving` into `@Public` in this kind of
guidelines/automated checks?
Piotrek
pt., 15 sty 2021 o 13:56 Timo Walther <twal...@apache.org>
napisał(a):
Hi everyone,
I would like to start a discussion how we treat deprecated and
legacy
code in Flink in the future. During the last years, our code base
has
grown quite a bit and a couple of interfaces and components have
been
reworked on the way.
I'm sure each component has a few legacy parts that are waiting
for
removal. Apart from keeping outdated API around for a couple of
releases
until users have updated their code, it is also often easier to
just
put
a @Deprecation annotation and postpone the actual change.
When looking at the current code, we have duplicated SQL planners,
duplicated APIs (DataSet/DataStream), duplicated source/sink
interfaces,
outdated connectors (Elasticsearch 5?) and dependencies (Scala
2.11?).
I'm wondering whether we should come up with some
legacy/deprecation
guidelines for the future.
Some examples:
- I could imagine new Flink-specific annotations for documenting
(in
code) in which version an interface was deprecated and when the
planned
removal should take place.
- Or guidelines that we drop a connector when the external project
does
not maintain the version for 6 months etc.
Plannable removal dates should also help users to not be surprised
when
a connector or Scala version is not supported anymore.
What do you think? I'm very happy to hear more opinions.
Regards,
Timo
