What JD and Josh said resonates also with the thoughts I had. On Tue, 10 Dec 2024 at 12:46, Caleb Rackliffe <calebrackli...@gmail.com> wrote:
> +1 to Josh's refinement of JD's proposal > > On Tue, Dec 10, 2024 at 11:42 AM Josh McKenzie <jmcken...@apache.org> > wrote: > >> +1 to this classification with one addition. I think we need to augment >> this with formalization on what we do with features we don't recommend >> people use (i.e. MV in their current incarnation). For something >> retroactively found to be unstable, we could add an "Unstable" >> qualification for it, leaving us with: >> >> >> 1. *Unstable: *Warnings on use, clearly communicated as to why, >> either on-track to be fixed or removed from the codebase. No lingering for >> years in a fugue state. *We should target never needing this >> classification.* >> 2. *Preview: *Ready to be tried by end users but has caveats and most >> likely is not api stable. Developer only documentation acceptable. >> 3. *Beta: *Feature complete/API stable but has not had enough testing >> to be considered rock solid. Developer and User documentation required. >> 4. *GA: *Ready for use, no known issue, PMC is satisfied with the >> testing that has been done >> >> >> To walk through how some of the flow might look to test the above: >> >> *Simple case:* >> *- *Preview -> Beta -> GA >> >> *Late discovered defect case:* >> *- *Preview -> Beta -> Unstable -> Beta -> GA >> >> *Pathological worst-case (i.e. MV):* >> - Preview -> Beta -> GA -> Unstable -> [Preview|Removed] >> >> On Tue, Dec 10, 2024, at 12:29 PM, Jeremiah Jordan wrote: >> >> I agree with Aleksey and Patrick. We should define terminology and then >> stick to it. My preferred list would be: >> >> >> 1. Preview - Ready to be tried by end users but has caveats and most >> likely is not api stable. >> 2. Beta - Feature complete/API stable but has not had enough testing >> to be considered rock solid. >> 3. GA - Ready for use, no known issue, PMC is satisfied with the >> testing that has been done >> >> >> Whether or not something is enabled by default or the default >> implementation is a separate access from the readiness. Though if we are >> replacing an existing thing with a new default I would hope we apply extra >> rigor to allowing that to happen. >> >> -Jeremiah >> >> On Dec 10, 2024 at 11:15:37 AM, Patrick McFadin <pmcfa...@gmail.com> >> wrote: >> >> I'm going to try to pull this back from the inevitable bikeshedding >> and airing of grievances that happen. Rewind all the way back to >> Josh's original point, which is a defined process. Why I really love >> this being brought up is our maturing process of communicating to the >> larger user base. The dev list has very few participants. Less than >> 1000 last I looked. Most users I talk to just want to know what they >> are getting. Well-formed, clear communication is how the PMC can let >> end users know that a new feature is one of three states: >> >> 1. Beta >> 2. Generally Available >> 3. Default (where appropriate) >> >> Yes! The work is just sorting out what each level means and then >> codifying that in confluence. Then, we look at any features that are >> under question, assign a level, and determine what it takes to go from >> one state to another. >> >> The CEPs need to reflect this change. What makes a Beta, GA, Default >> for new feature X. It makes it clear for implementers and end users, >> which is an important feature of project maturity. >> >> Patrick >> >> >> >> On Dec 10, 2024 at 5:46:38 AM, Aleksey Yeshchenko <alek...@apple.com> >> wrote: >> >> What we’ve done is we’ve overloaded the term ‘experimental’ to mean too >> many related but different ideas. We need additional, more specific >> terminology to disambiguate. >> >> 1. Labelling released features that were known to be unstable at release >> as ‘experimental’ retroactively shouldn’t happen and AFAIK only happened >> once, with MVs, and ‘experimental’ there was just a euphemism for ‘broken’. >> Our practices are more mature now, I like to think, that a situation like >> this would not arise in the future - the bar for releasing a completed >> marketable feature is higher. So the label ‘experimental’ should not be >> applied retroactively to anything. >> >> 2. It’s possible that a released, once considered production-ready >> feature, might be discovered to be deeply flawed after being released >> already. We need to temporarily mark such a feature as ‘broken' or >> ‘flawed'. Not experimental, and not even ‘unstable’. Make sure we emit a >> warning on its use everywhere, and, if possible, make it opt-in in the next >> major, at the very least, to prevent new uses of it. Announce on dev, add a >> note in NEWS.txt, etc. If the flaws are later addressed, remove the label. >> Removing the feature itself might not be possible, but should be >> considered, with heavy advanced telegraphing to the community. >> >> 3. There is probably room for genuine use of ‘experimental’ as a feature >> label. For opt-in features that we commit with an understanding that they >> might not make it at all. Unstable API is implied here, but a feature can >> also have an unstable API without being experimental - so ‘experimental' >> doesn’t equal to ‘api-unstable’. These should not be relied on by any >> production code, they would be heavily gated by unambiguous configuration >> flags, disabled by default, allowed to be removed or changed in any version >> including a minor one. >> >> 4. New features without known flaws, intended to be production-ready and >> marketable eventually, that we may want to gain some real-world confidence >> with before we are happy to market or make default. UCS, for example, which >> seems to be in heavy use in Astra and doesn’t have any known open issues >> (AFAIK). It’s not experimental, it’s not unstable, it’s not ‘alpha’ or >> ‘beta’, it just hasn't been widely enough used to have gained a lot of >> confidence. It’s just new. I’m not sure what label even applies here. It’s >> just a regular feature that happens to be new, doesn’t need a label, just >> needs to see some widespread use before we can make it a default. No other >> limitation on its use. >> >> 5. Early-integrated, not-yet fully-completed features that are NOT >> experimental in nature. Isolated, gated behind deep configuration flags. >> Have a CEP behind them, we trust that they will be eventually completed, >> but for pragmatic reasons it just made sense to commit them at an earlier >> stage. ‘Preview’, ‘alpha’, ‘beta’ are labels that could apply here >> depending on current feature readiness status. API-instability is implied. >> Once finished they just become a regular new feature, no flag needed, no >> heavy config gating needed. >> >> I might be missing some scenarios here. >> >> >> >> >>
