+1 to Josh's refinement of JD's proposal On Tue, Dec 10, 2024 at 11:42 AM Josh McKenzie <jmcken...@apache.org> wrote:
> +1 to this classification with one addition. I think we need to augment > this with formalization on what we do with features we don't recommend > people use (i.e. MV in their current incarnation). For something > retroactively found to be unstable, we could add an "Unstable" > qualification for it, leaving us with: > > > 1. *Unstable: *Warnings on use, clearly communicated as to why, either > on-track to be fixed or removed from the codebase. No lingering for years > in a fugue state. *We should target never needing this classification.* > 2. *Preview: *Ready to be tried by end users but has caveats and most > likely is not api stable. Developer only documentation acceptable. > 3. *Beta: *Feature complete/API stable but has not had enough testing > to be considered rock solid. Developer and User documentation required. > 4. *GA: *Ready for use, no known issue, PMC is satisfied with the > testing that has been done > > > To walk through how some of the flow might look to test the above: > > *Simple case:* > *- *Preview -> Beta -> GA > > *Late discovered defect case:* > *- *Preview -> Beta -> Unstable -> Beta -> GA > > *Pathological worst-case (i.e. MV):* > - Preview -> Beta -> GA -> Unstable -> [Preview|Removed] > > On Tue, Dec 10, 2024, at 12:29 PM, Jeremiah Jordan wrote: > > I agree with Aleksey and Patrick. We should define terminology and then > stick to it. My preferred list would be: > > > 1. Preview - Ready to be tried by end users but has caveats and most > likely is not api stable. > 2. Beta - Feature complete/API stable but has not had enough testing > to be considered rock solid. > 3. GA - Ready for use, no known issue, PMC is satisfied with the > testing that has been done > > > Whether or not something is enabled by default or the default > implementation is a separate access from the readiness. Though if we are > replacing an existing thing with a new default I would hope we apply extra > rigor to allowing that to happen. > > -Jeremiah > > On Dec 10, 2024 at 11:15:37 AM, Patrick McFadin <pmcfa...@gmail.com> > wrote: > > I'm going to try to pull this back from the inevitable bikeshedding > and airing of grievances that happen. Rewind all the way back to > Josh's original point, which is a defined process. Why I really love > this being brought up is our maturing process of communicating to the > larger user base. The dev list has very few participants. Less than > 1000 last I looked. Most users I talk to just want to know what they > are getting. Well-formed, clear communication is how the PMC can let > end users know that a new feature is one of three states: > > 1. Beta > 2. Generally Available > 3. Default (where appropriate) > > Yes! The work is just sorting out what each level means and then > codifying that in confluence. Then, we look at any features that are > under question, assign a level, and determine what it takes to go from > one state to another. > > The CEPs need to reflect this change. What makes a Beta, GA, Default > for new feature X. It makes it clear for implementers and end users, > which is an important feature of project maturity. > > Patrick > > > > On Dec 10, 2024 at 5:46:38 AM, Aleksey Yeshchenko <alek...@apple.com> > wrote: > > What we’ve done is we’ve overloaded the term ‘experimental’ to mean too > many related but different ideas. We need additional, more specific > terminology to disambiguate. > > 1. Labelling released features that were known to be unstable at release > as ‘experimental’ retroactively shouldn’t happen and AFAIK only happened > once, with MVs, and ‘experimental’ there was just a euphemism for ‘broken’. > Our practices are more mature now, I like to think, that a situation like > this would not arise in the future - the bar for releasing a completed > marketable feature is higher. So the label ‘experimental’ should not be > applied retroactively to anything. > > 2. It’s possible that a released, once considered production-ready > feature, might be discovered to be deeply flawed after being released > already. We need to temporarily mark such a feature as ‘broken' or > ‘flawed'. Not experimental, and not even ‘unstable’. Make sure we emit a > warning on its use everywhere, and, if possible, make it opt-in in the next > major, at the very least, to prevent new uses of it. Announce on dev, add a > note in NEWS.txt, etc. If the flaws are later addressed, remove the label. > Removing the feature itself might not be possible, but should be > considered, with heavy advanced telegraphing to the community. > > 3. There is probably room for genuine use of ‘experimental’ as a feature > label. For opt-in features that we commit with an understanding that they > might not make it at all. Unstable API is implied here, but a feature can > also have an unstable API without being experimental - so ‘experimental' > doesn’t equal to ‘api-unstable’. These should not be relied on by any > production code, they would be heavily gated by unambiguous configuration > flags, disabled by default, allowed to be removed or changed in any version > including a minor one. > > 4. New features without known flaws, intended to be production-ready and > marketable eventually, that we may want to gain some real-world confidence > with before we are happy to market or make default. UCS, for example, which > seems to be in heavy use in Astra and doesn’t have any known open issues > (AFAIK). It’s not experimental, it’s not unstable, it’s not ‘alpha’ or > ‘beta’, it just hasn't been widely enough used to have gained a lot of > confidence. It’s just new. I’m not sure what label even applies here. It’s > just a regular feature that happens to be new, doesn’t need a label, just > needs to see some widespread use before we can make it a default. No other > limitation on its use. > > 5. Early-integrated, not-yet fully-completed features that are NOT > experimental in nature. Isolated, gated behind deep configuration flags. > Have a CEP behind them, we trust that they will be eventually completed, > but for pragmatic reasons it just made sense to commit them at an earlier > stage. ‘Preview’, ‘alpha’, ‘beta’ are labels that could apply here > depending on current feature readiness status. API-instability is implied. > Once finished they just become a regular new feature, no flag needed, no > heavy config gating needed. > > I might be missing some scenarios here. > > > > >
