I think it would be better to avoid 'alpha' since we do beta releases,
and I agree with Aleksey that we'd be overloading 'alpha' and perhaps
causing confusion.
Kind Regards,
Brandon
On Thu, Dec 12, 2024 at 8:58 AM Benedict <bened...@apache.org> wrote:
>
> I think alpha is fine. It communicates fairly well that there’s no near term
> expectation they will be production capable.
>
> There is (I think) still an intention to improve them, but they are janky. If
> we don’t intend to begin fixing the feature within the next year or so we
> should deprecate it entirely.
>
>
> On 12 Dec 2024, at 14:46, Aleksey Yeshchenko <alek...@apple.com> wrote:
>
> But MVs are not alpha or preview, as they are not actively being worked on.
> They are currently broken. Calling them ‘alpha’ makes ‘alpha’ overloaded and
> less useful.
>
> On 12 Dec 2024, at 14:00, Josh McKenzie <jmcken...@apache.org> wrote:
>
> But we also need an approved non-euphemism for features like MVs (I suggest
> ‘broken’) and possibly a softer version of it ('dangerous') for our existing
> features that work fine in some narrow well-defined circumstances but will
> blow in your face if you don’t know exactly what you are doing.
>
> Feels like the real answer is:
>
> Endeavor to never get ourselves into this state
> Take immediate action if we discover we're there (fix feature if possible,
> deprecate and remove if not). Not "leave to fester for years"
>
> I like the introduction of 'alpha' as an alias for 'Preview'; not sure why
> that wasn't what we immediately came up with collectively given how
> widespread its usage is. :)
>
> What would demoting MV's to 'alpha' right now look like? We'd warn on their
> usage w/some different structure and verbiage, and it'd be pretty implicitly
> clear to people they shouldn't use it in production right?
>
> It seems to me that the 3 categories would be sufficient even to handle our
> current scenario where we have some things in the system that are a Bad Idea
> to use in production.
>
> On Thu, Dec 12, 2024, at 6:06 AM, Aleksey Yeshchenko wrote:
>
> I don’t like ‘unstable’ either, albeit for a different reason, but I don’t
> think three is enough and fits, as we already have some features that don’t
> fit into either of (preview,beta,ga) - released but broken, released but
> dangerous, deprecated, removed.
>
> For new features going forward, alpha (preview) -> beta -> GA works well
> enough.
>
> But we also need an approved non-euphemism for features like MVs (I suggest
> ‘broken’) and possibly a softer version of it ('dangerous') for our existing
> features that work fine in some narrow well-defined circumstances but will
> blow in your face if you don’t know exactly what you are doing.
>
> These classifications are largely orthogonal.
>
> Alpha(preview)->Beta->GA communicates readiness of a feature under
> development, with GA being the default final state for most features.
>
> From there a feature can transition into ‘broken’ or ‘dangerous’ territory.
> Serious issues get uncovered (very) late sometimes. It is what it is.
> And we do deprecate and remove functionality when it’s superseded.
>
>
> -1 on unstable. It's way too many words than are needed. Three is a
> magic number and fits:
>
> Preview
> Beta
> GA
>
>
> On 11 Dec 2024, at 18:50, Josh McKenzie <jmcken...@apache.org> wrote:
>
> A structured, disciplined approach to graduating something from [Optional] ->
> [Default] makes sense to me, similar to how we're talking about a structured
> flow of [Preview] -> [Beta] -> [GA]. Having those clear stages gives us a
> framework to define what requirements of stage transitions would be which'll
> ideally lead to us producing higher quality, more predictable, more
> consistent results for our end users.
>
> For instance, requirements from [Optional] -> [Default] could be higher level
> abstractions like:
>
> Confidence in stability
> Strong evidence to indicate superiority in majority of workloads (by count or
> importance or size, etc)
>
> These are all things we kind of do implicitly and ad-hoc on the mailing list,
> and I'm not looking to tie us down to any granular structure or specificity.
> More thinking it could be useful for someone that's worked on something who
> wonders "Huh. How do I take this from being optional to the default?" and
> having an answer better than "reinvent the wheel every time and fling
> spaghetti at the dev list and pray".
>
> :)
>
>
> On Wed, Dec 11, 2024, at 1:04 PM, Paulo Motta wrote:
>
> Thanks for bringing up this topic, Josh.
>
> Outside of the major features (ie. MV/SAI/TCM/Accord), one related discussion
> in this topic is: how can we "promote" small improvements in existing
> features from optional to default ?
>
> It makes sense to have optimizations launched behind a feature flag initially
> (beta phase) while the improvement gets real world exposure, but I think we
> need a better way to promote these optimizations to default behavior on a
> regular cadence.
>
> Take for example optimized repairs from CASSANDRA-16274. It was launched in
> 4.x as an optional feature gated behind a flag, ie.
> auto_optimise_full_repair_streams: false.
>
> I could be easily missing something, but is there a world where non-optimized
> repairs make sense once this optimization is proven to work ? I agree this is
> fine while the feature is maturing, but at some point we need to rip the
> bandaid and make the optimization default (and clearly communicate that).
> This would allow cleanup code toil of default behavior that is no longer
> being used, because everyone is enabling the improvement during deployment.
>
> This is just one example to demonstrate the issue and I don't want this
> discussion to focus on this particular case, but I can think of other
> improvements launched as optional that are never made default.
>
> I don't know if this should be continued to be addressed on a
> improvement-by-improvement basis or if we could have a more streamlined
> process to review and communicate these changes more consciously at every
> major release.
>
> In the same way we open a loop when adding an optimized behavior behind a
> feature flag, I think we should have a process to close these loops by
> promoting these optimizations to default when it makes sense.
>
> On Tue, Dec 10, 2024 at 2:10 PM Josh McKenzie <jmcken...@apache.org> wrote:
>
>
> So some questions to test a world w/3 classifications (Preview, Beta, GA):
> - What would we do with the current experimental features (MV's, JDK17,
> witnesses, etc)? Flag them as preview or beta as appropriate on a
> case-by-case basis and add runtime warnings / documentation where missing?
>
> - What would we do in the future if a feature's GA and we discover a Very Big
> Problem with it that'll take some time to fix? Keep it GA but cut a hotfix
> release w/a bunch of warnings? Bounce it back to Preview? Leave it be and
> just feverishly try and fix it?
>
> for policy decisions like this (that don’t need to be agreed in advance) we
> should try to legislate the minimum necessary policy to proceed today
>
> Definitely agree; MV's being in limbo for years strains the "3-step
> classification" structure for me. If we want to avoid having a solution for
> the MV-shaped case on the grounds we won't allow ourselves to reach this
> state again in the future, that seems reasonable. With the caveat that we
> might be in a similar situation with vector search right now, etc.
>
>
> On Tue, Dec 10, 2024, at 1:48 PM, Benedict Elliott Smith wrote:
>
> Yep, I agree with this - we can revisit if we ever absolutely feel the need
> to add additional states for exceptional circumstances.
>
> > On 10 Dec 2024, at 13:24, Patrick McFadin <pmcfa...@gmail.com> wrote:
> >
> > -1 on unstable. It's way too many words than are needed. Three is a
> > magic number and fits:
> >
> > Preview
> > Beta
> > GA
> >
> > As a matter of testing the process, any pending CEP should go though
> > this exercise so we can see how it will work.
> >
> > PS
> > Got the actual numbers from Whimsy.
> > DEV - 1425 users
> > USER - 2650
> >
> > This means that when features experience a state change, finding more
> > avenues to get the word out will be important.
> >
> > On Tue, Dec 10, 2024 at 10:04 AM Benedict Elliott Smith
> > <bened...@apache.org> wrote:
> >>
> >> As an aside, it would be nice to admit we basically revisit everything
> >> each time it becomes relevant again, and for policy decisions like this
> >> (that don’t need to be agreed in advance) we should try to legislate the
> >> minimum necessary policy to proceed today, and leave future refinements
> >> for later when the relevant context arises.
> >>
> >> On 10 Dec 2024, at 13:00, Benedict Elliott Smith <bened...@apache.org>
> >> wrote:
> >>
> >> I agree with Aleksey that if we think something is broken, we shouldn’t
> >> use euphemisms, and for this reason I don’t like unstable (this could for
> >> instance simply mean API unstable). If we intend to never need this
> >> descriptor, we should avoid bike-shedding and insert a “placeholder” for
> >> now to be refined as and when we need it when we have the necessary future
> >> context.
> >>
> >> i.e.
> >>
> >> preview -> beta -> [“has problems that will take time to resolve
> >> placeholder” -> beta] -> GA
> >>
> >>
> >>
> >> On 10 Dec 2024, at 12:39, Josh McKenzie <jmcken...@apache.org> wrote:
> >>
> >> +1 to this classification with one addition. I think we need to augment
> >> this with formalization on what we do with features we don't recommend
> >> people use (i.e. MV in their current incarnation). For something
> >> retroactively found to be unstable, we could add an "Unstable"
> >> qualification for it, leaving us with:
> >>
> >> Unstable: Warnings on use, clearly communicated as to why, either on-track
> >> to be fixed or removed from the codebase. No lingering for years in a
> >> fugue state. We should target never needing this classification.
> >> Preview: Ready to be tried by end users but has caveats and most likely is
> >> not api stable. Developer only documentation acceptable.
> >> Beta: Feature complete/API stable but has not had enough testing to be
> >> considered rock solid. Developer and User documentation required.
> >> GA: Ready for use, no known issue, PMC is satisfied with the testing that
> >> has been done
> >>
> >>
> >> To walk through how some of the flow might look to test the above:
> >>
> >> Simple case:
> >> - Preview -> Beta -> GA
> >>
> >> Late discovered defect case:
> >> - Preview -> Beta -> Unstable -> Beta -> GA
> >>
> >> Pathological worst-case (i.e. MV):
> >> - Preview -> Beta -> GA -> Unstable -> [Preview|Removed]
> >>
> >> On Tue, Dec 10, 2024, at 12:29 PM, Jeremiah Jordan wrote:
> >>
> >> I agree with Aleksey and Patrick. We should define terminology and then
> >> stick to it. My preferred list would be:
> >>
> >> Preview - Ready to be tried by end users but has caveats and most likely
> >> is not api stable.
> >> Beta - Feature complete/API stable but has not had enough testing to be
> >> considered rock solid.
> >> GA - Ready for use, no known issue, PMC is satisfied with the testing that
> >> has been done
> >>
> >>
> >> Whether or not something is enabled by default or the default
> >> implementation is a separate access from the readiness. Though if we are
> >> replacing an existing thing with a new default I would hope we apply extra
> >> rigor to allowing that to happen.
> >>
> >> -Jeremiah
> >>
> >> On Dec 10, 2024 at 11:15:37 AM, Patrick McFadin <pmcfa...@gmail.com> wrote:
> >>
> >> I'm going to try to pull this back from the inevitable bikeshedding
> >> and airing of grievances that happen. Rewind all the way back to
> >> Josh's original point, which is a defined process. Why I really love
> >> this being brought up is our maturing process of communicating to the
> >> larger user base. The dev list has very few participants. Less than
> >> 1000 last I looked. Most users I talk to just want to know what they
> >> are getting. Well-formed, clear communication is how the PMC can let
> >> end users know that a new feature is one of three states:
> >>
> >> 1. Beta
> >> 2. Generally Available
> >> 3. Default (where appropriate)
> >>
> >> Yes! The work is just sorting out what each level means and then
> >> codifying that in confluence. Then, we look at any features that are
> >> under question, assign a level, and determine what it takes to go from
> >> one state to another.
> >>
> >> The CEPs need to reflect this change. What makes a Beta, GA, Default
> >> for new feature X. It makes it clear for implementers and end users,
> >> which is an important feature of project maturity.
> >>
> >> Patrick
> >>
> >>
> >>
> >> On Dec 10, 2024 at 5:46:38 AM, Aleksey Yeshchenko <alek...@apple.com>
> >> wrote:
> >>
> >> What we’ve done is we’ve overloaded the term ‘experimental’ to mean too
> >> many related but different ideas. We need additional, more specific
> >> terminology to disambiguate.
> >>
> >> 1. Labelling released features that were known to be unstable at release
> >> as ‘experimental’ retroactively shouldn’t happen and AFAIK only happened
> >> once, with MVs, and ‘experimental’ there was just a euphemism for
> >> ‘broken’. Our practices are more mature now, I like to think, that a
> >> situation like this would not arise in the future - the bar for releasing
> >> a completed marketable feature is higher. So the label ‘experimental’
> >> should not be applied retroactively to anything.
> >>
> >> 2. It’s possible that a released, once considered production-ready
> >> feature, might be discovered to be deeply flawed after being released
> >> already. We need to temporarily mark such a feature as ‘broken' or
> >> ‘flawed'. Not experimental, and not even ‘unstable’. Make sure we emit a
> >> warning on its use everywhere, and, if possible, make it opt-in in the
> >> next major, at the very least, to prevent new uses of it. Announce on dev,
> >> add a note in NEWS.txt, etc. If the flaws are later addressed, remove the
> >> label. Removing the feature itself might not be possible, but should be
> >> considered, with heavy advanced telegraphing to the community.
> >>
> >> 3. There is probably room for genuine use of ‘experimental’ as a feature
> >> label. For opt-in features that we commit with an understanding that they
> >> might not make it at all. Unstable API is implied here, but a feature can
> >> also have an unstable API without being experimental - so ‘experimental'
> >> doesn’t equal to ‘api-unstable’. These should not be relied on by any
> >> production code, they would be heavily gated by unambiguous configuration
> >> flags, disabled by default, allowed to be removed or changed in any
> >> version including a minor one.
> >>
> >> 4. New features without known flaws, intended to be production-ready and
> >> marketable eventually, that we may want to gain some real-world confidence
> >> with before we are happy to market or make default. UCS, for example,
> >> which seems to be in heavy use in Astra and doesn’t have any known open
> >> issues (AFAIK). It’s not experimental, it’s not unstable, it’s not ‘alpha’
> >> or ‘beta’, it just hasn't been widely enough used to have gained a lot of
> >> confidence. It’s just new. I’m not sure what label even applies here. It’s
> >> just a regular feature that happens to be new, doesn’t need a label, just
> >> needs to see some widespread use before we can make it a default. No other
> >> limitation on its use.
> >>
> >> 5. Early-integrated, not-yet fully-completed features that are NOT
> >> experimental in nature. Isolated, gated behind deep configuration flags.
> >> Have a CEP behind them, we trust that they will be eventually completed,
> >> but for pragmatic reasons it just made sense to commit them at an earlier
> >> stage. ‘Preview’, ‘alpha’, ‘beta’ are labels that could apply here
> >> depending on current feature readiness status. API-instability is implied.
> >> Once finished they just become a regular new feature, no flag needed, no
> >> heavy config gating needed.
> >>
> >> I might be missing some scenarios here.
> >>
> >>
> >>
>
>
