I like that we are expanding annotations in the hopes of balancing safety and freedom and it's helpful to see the other projects.
My confession 🙈🙉🙊 is that I stopped using the Experimental annotation because I reasoned that there's already a license saying "don't sue us, it's your fault". I stopped using Internal because knowing people are people that when you expose something in the code, and people can use it, they will ignore the Internal? Also, I reasoned if I really don't want others to use it, I would make it package private or private. But I am going to be more diligent now that I know these annotations are getting attention On Fri, Mar 31, 2023, 2:05 PM Kenneth Knowles <k...@apache.org> wrote: > Hi all, > > Long ago, we adopted two annotations in Beam to communicate to users: > > - `@Experimental` indicates that an API might change > - `@Internal` indicates that an API is not meant for users. > > I've seen some real problems with this approach: > > - Users are afraid to use `@Experimental` APIs, because they are worried > they are not production-ready. But it really just means they might change, > and has nothing to do with that. > - People write new code and do not put `@Experimental` annotations on it, > even though it really should be able to change for a while, so we can do a > good job. > - I'm seeing a culture of being afraid to change things, even when it > would be good for users, because our API surface area is far too large and > not explicitly chosen. > - `@Internal` is not that well-known. And now we have many target > audiences: Beam devs, PTransform devs, tool devs, pipeline authors. Some of > them probably want to use `@Internal` stuff! > > I looked at a couple sibling projects and what they have > - Flink: > - Spark: > > They have many more tags, and some of them seem to have reverse defaults > to Beam. > > Flink: > https://github.com/apache/flink/tree/master/flink-annotations/src/main/java/org/apache/flink/annotation > > - Experimental > - Internal.java > - Public > - PublicEvolving > - VisibleForTesting > > Spark: > https://github.com/apache/spark/tree/master/common/tags/src/main/java/org/apache/spark/annotation > and > https://github.com/apache/spark/tree/master/common/tags/src/main/scala/org/apache/spark/annotation > > - AlphaComponent > - DeveloperApi > - Evolving > - Experimental > - Private > - Stable > - Unstable > - Since > > I think it would help users to understand Beam with some simple, though > possibly large-scale changes. My goal would be: > > - new code is changeable/evolving by default (so we don't have to always > remember to annotate it) but users have confidence they can use it in > production (because we have good software engineering practices) > - Experimental would be reserved for more risky things > - after we are confident an API is stable, because it has been the same > across a couple releases, we mark it > > A concrete proposal to achieve this would be: > > - Add a @Stable annotation and use it as appropriate on our primary APIs > - [Possibly] add an @Evolving annotation that would also be the default. > - Remove most `@Experimental` annotations or change them to `@Evolving` > - Communicate about this (somehow). If possible, surface the `@Evolving` > default in documentation. > > The last bit is the hardest. > > Kenn >
