+1 to this classification with one addition. I think we need to augment this with formalization on what we do with features we don't recommend people use (i.e. MV in their current incarnation). For something retroactively found to be unstable, we could add an "Unstable" qualification for it, leaving us with:
1. *Unstable: *Warnings on use, clearly communicated as to why, either on-track to be fixed or removed from the codebase. No lingering for years in a fugue state. *We should target never needing this classification.* 2. *Preview: *Ready to be tried by end users but has caveats and most likely is not api stable. Developer only documentation acceptable. 3. *Beta: *Feature complete/API stable but has not had enough testing to be considered rock solid. Developer and User documentation required. 4. *GA: *Ready for use, no known issue, PMC is satisfied with the testing that has been done To walk through how some of the flow might look to test the above: *Simple case:* *- *Preview -> Beta -> GA *Late discovered defect case:* *- *Preview -> Beta -> Unstable -> Beta -> GA *Pathological worst-case (i.e. MV):* - Preview -> Beta -> GA -> Unstable -> [Preview|Removed] On Tue, Dec 10, 2024, at 12:29 PM, Jeremiah Jordan wrote: > I agree with Aleksey and Patrick. We should define terminology and then > stick to it. My preferred list would be: > > 1. Preview - Ready to be tried by end users but has caveats and most likely > is not api stable. > 2. Beta - Feature complete/API stable but has not had enough testing to be > considered rock solid. > 3. GA - Ready for use, no known issue, PMC is satisfied with the testing > that has been done > > Whether or not something is enabled by default or the default implementation > is a separate access from the readiness. Though if we are replacing an > existing thing with a new default I would hope we apply extra rigor to > allowing that to happen. > > -Jeremiah > > On Dec 10, 2024 at 11:15:37 AM, Patrick McFadin <pmcfa...@gmail.com> wrote: >> I'm going to try to pull this back from the inevitable bikeshedding >> and airing of grievances that happen. Rewind all the way back to >> Josh's original point, which is a defined process. Why I really love >> this being brought up is our maturing process of communicating to the >> larger user base. The dev list has very few participants. Less than >> 1000 last I looked. Most users I talk to just want to know what they >> are getting. Well-formed, clear communication is how the PMC can let >> end users know that a new feature is one of three states: >> >> 1. Beta >> 2. Generally Available >> 3. Default (where appropriate) >> >> Yes! The work is just sorting out what each level means and then >> codifying that in confluence. Then, we look at any features that are >> under question, assign a level, and determine what it takes to go from >> one state to another. >> >> The CEPs need to reflect this change. What makes a Beta, GA, Default >> for new feature X. It makes it clear for implementers and end users, >> which is an important feature of project maturity. >> >> Patrick > > > On Dec 10, 2024 at 5:46:38 AM, Aleksey Yeshchenko <alek...@apple.com> wrote: >> What we’ve done is we’ve overloaded the term ‘experimental’ to mean too many >> related but different ideas. We need additional, more specific terminology >> to disambiguate. >> >> 1. Labelling released features that were known to be unstable at release as >> ‘experimental’ retroactively shouldn’t happen and AFAIK only happened once, >> with MVs, and ‘experimental’ there was just a euphemism for ‘broken’. Our >> practices are more mature now, I like to think, that a situation like this >> would not arise in the future - the bar for releasing a completed marketable >> feature is higher. So the label ‘experimental’ should not be applied >> retroactively to anything. >> >> 2. It’s possible that a released, once considered production-ready feature, >> might be discovered to be deeply flawed after being released already. We >> need to temporarily mark such a feature as ‘broken' or ‘flawed'. Not >> experimental, and not even ‘unstable’. Make sure we emit a warning on its >> use everywhere, and, if possible, make it opt-in in the next major, at the >> very least, to prevent new uses of it. Announce on dev, add a note in >> NEWS.txt, etc. If the flaws are later addressed, remove the label. Removing >> the feature itself might not be possible, but should be considered, with >> heavy advanced telegraphing to the community. >> >> 3. There is probably room for genuine use of ‘experimental’ as a feature >> label. For opt-in features that we commit with an understanding that they >> might not make it at all. Unstable API is implied here, but a feature can >> also have an unstable API without being experimental - so ‘experimental' >> doesn’t equal to ‘api-unstable’. These should not be relied on by any >> production code, they would be heavily gated by unambiguous configuration >> flags, disabled by default, allowed to be removed or changed in any version >> including a minor one. >> >> 4. New features without known flaws, intended to be production-ready and >> marketable eventually, that we may want to gain some real-world confidence >> with before we are happy to market or make default. UCS, for example, which >> seems to be in heavy use in Astra and doesn’t have any known open issues >> (AFAIK). It’s not experimental, it’s not unstable, it’s not ‘alpha’ or >> ‘beta’, it just hasn't been widely enough used to have gained a lot of >> confidence. It’s just new. I’m not sure what label even applies here. It’s >> just a regular feature that happens to be new, doesn’t need a label, just >> needs to see some widespread use before we can make it a default. No other >> limitation on its use. >> >> 5. Early-integrated, not-yet fully-completed features that are NOT >> experimental in nature. Isolated, gated behind deep configuration flags. >> Have a CEP behind them, we trust that they will be eventually completed, but >> for pragmatic reasons it just made sense to commit them at an earlier stage. >> ‘Preview’, ‘alpha’, ‘beta’ are labels that could apply here depending on >> current feature readiness status. API-instability is implied. Once finished >> they just become a regular new feature, no flag needed, no heavy config >> gating needed. >> >> I might be missing some scenarios here. > >
