I'm certainly a fan of docker images as a dev tool! LorinaSent from my Verizon,
Samsung Galaxy smartphone
-------- Original message --------From: Patrick McFadin <pmcfa...@gmail.com>
Date: 10/24/23 13:31 (GMT-08:00) To: dev@cassandra.apache.org Subject: Re:
Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)
Let me make that really easy. Hell yesNot everybody runs CCM, I've tried but
I've met resistance. Compiling your own version usually involves me saying the
words "Yes, ant realclean exists. I'm not trolling you" docker pull <image>
works on every OS and curates a single node experience. On Tue, Oct 24, 2023 at
12:37 PM Josh McKenzie <jmcken...@apache.org> wrote:In order for the project to
advertise the release outside the dev@ list it needs to be a formal
release.That's my reading as
well:https://www.apache.org/legal/release-policy.html#release-definitionI
wonder if there'd be value in us having a cronned job that'd do nightly docker
container builds on trunk + feature branches, archived for N days, and we make
that generally known to the dev@ list here so folks that want to poke at the
current state of trunk or other branches could do so with very low friction.
We'd probably see more engagement on feature branches if it was turn-key easy
for other C* devs to spin the up and check them out.For what you're talking
about here Patrick (a docker image for folks outside the dev@ audience and more
user-facing), we'd want to vote on it and go through the formal process.On Tue,
Oct 24, 2023, at 3:10 PM, Jeremiah Jordan wrote:In order for the project to
advertise the release outside the dev@ list it needs to be a formal release.
That just means that there was a release vote and at least 3 PMC members +1’ed
it, and there are more +1 than there are -1, and we follow all the normal
release rules. The ASF release process doesn’t care what branch you cut the
artifacts from or what version you call it.So the project can cut artifacts for
and release a 5.1-alpha1, 5.1-dev-preview1, what ever we want to version this
thing, from trunk or any other branch name we want.-JeremiahOn Oct 24, 2023 at
2:03:41 PM, Patrick McFadin <pmcfa...@gmail.com> wrote:I would like to have
something for developers to use ASAP to try the Accord syntax. Very few people
have seen it, and I think there's a learning curve we can start earlier.It's my
understanding that ASF policy is that it needs to be a project release to
create a docker image.On Tue, Oct 24, 2023 at 11:54 AM Jeremiah Jordan
<jeremiah.jor...@gmail.com> wrote:If we decide to go the route of not merging
TCM to the 5.0 branch. Do we actually need to immediately cut a 5.1 branch?
Can we work on stabilizing things while it is in trunk and cut the 5.1 branch
when we actually think we are near releasing? I don’t see any reason we can
not cut “preview” artifacts from trunk?-JeremiahOn Oct 24, 2023 at 11:54:25 AM,
Jon Haddad <rustyrazorbl...@apache.org> wrote:I guess at the end of the day,
shipping a release with a bunch of awesome features is better than holding it
back. If there's 2 big releases in 6 months the community isn't any worse off.
We either ship something, or nothing, and something is probably better.JonOn
2023/10/24 16:27:04 Patrick McFadin wrote:+1 to what you are saying, Josh.
Based on the last survey, yes, everyonewas excited about Accord, but SAI and
UCS were pretty high on the list.Benedict and I had a good conversation last
night, and now I understandmore essential details for this conversation. TCM is
taking far more workthan initially scoped, and Accord depends on a stable TCM.
TCM is monthsbehind and that's a critical fact, and one I personally just
learned of. Ithought things were wrapping up this month, and we were in the
testingphase. I get why that's a topic we are dancing around. Nobody wants to
sayship dates are slipping because that's part of our culture.
It'sdisappointing and, if new information, an unwelcome surprise, but none ofus
should be angry or in a blamey mood because I guarantee every one of ushas
shipped the code late. My reaction yesterday was based on an
incorrectassumption. Now that I have a better picture, my point of view is
changing.Josh's point about what's best for users is crucial. Users deserve
stablecode with a regular cadence of features that make their lives easier. If
weput 5.0 on hold for TCM + Accord, users will get neither for a very longtime.
And I mentioned a disaster yesterday. A bigger disaster would beshipping Accord
with a major bug that causes data loss, eroding communitytrust. Accord has to
be the most bulletproof of all bulletproof features.The pressure to ship is
only going to increase and that's fertile groundfor that sort of bug.So, taking
a step back and with a clearer picture, I support the 5.0 + 5.1plan mainly
because I don't think 5.1 is (or should be) a fast follow.For the user
community, the communication should be straightforward. TCM +Accord are turning
out to be much more complicated than was originallyscoped, and for good
reasons. Our first principle is to provide a stableand reliable system, so as a
result, we'll be de-coupling TCM + Accord from5.0 into a 5.1 branch, which is
available in parallel to 5.0 whileadditional hardening and testing is done. We
can communicate this in a blogpost.,To make this much more palatable to our use
community, if we can get abuild and docker image available ASAP with Accord, it
will allow developersto start playing with the syntax. Up to this point, that
hasn't been widelyavailable unless you compile the code yourself. Developers
need tounderstand how this will work in an application, and up to this point,
thesyntax is text they see in my slides. We need to get some hands-on and
thatwill get our user community engaged on Accord this calendar year.
Thefeedback may even uncover some critical changes we'll need to make. Lack
ofaccess to Accord by developers is a critical problem we can fix soon andthere
will be plenty of excitement there and start building use casesbefore the final
code ships.I'm bummed but realistic. It sucks that I won't have a pony for
Christmas,but maybe one for my birthday?PatrickOn Tue, Oct 24, 2023 at 7:23 AM
Josh McKenzie <jmcken...@apache.org> wrote:> Maybe it won't be a glamorous
release but shipping> 5.0 mitigates our worst case scenario.>> I disagree with
this characterization of 5.0 personally. UCS, SAI, Trie> memtables and
sstables, maybe vector ANN if the sub-tasks on C-18715 are> accurate, all
combine to make 5.0 a pretty glamorous release IMO> independent of TCM and
Accord. Accord is a true paradigm-shift game-changer> so it's easy to think of
5.0 as uneventful in comparison, and TCM helps> resolve one of the biggest
pain-points in our system for over a decade, but> I think 5.0 is a very meaty
release in its own right today.>> Anyway - I agree with you Brandon re:
timelines. If things take longer> than we'd hope (which, if I think back, they
do roughly 100% of the time on> this project), blocking on these features could
both lead to a significant> delay in 5.0 going out as well as increasing
pressure and risk of burnout> on the folks working on it. While I believe we
all need some balanced> urgency to do our best work, being under the gun for
something with a hard> deadline or having an entire project drag along blocked
on you is not where> I want any of us to be.>> Part of why we talked about
going to primarily annual calendar-based> releases was to avoid precisely this
situation, where something that> *feels* right at the cusp of merging leads us
to delay a release> repeatedly. We discussed this a couple times this year:> 1:
https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3,> where Mick
proposed a "soft-freeze" for everything w/out exception and 1st> week October
"hard-freeze", and there was assumed to be lazy consensus> 2:
https://lists.apache.org/thread/mzj3dq8b7mzf60k6mkby88b9n9ywmsgw,> where we
kept along with what we discussed in 1 but added in CEP-30 to be> waivered in
as well.>> So. We're at a crossroads here where we need to either follow
through with> what we all agreed to earlier this year, or acknowledge that our
best> intention of calendar-based releases can't stand up to our optimism and>
desire to get these features into the next major.>> There's no immediate
obvious better path to me in terms of what's best for> our users. This is a
situation of risk tolerance with a lot of unknowns> that could go either way.>>
Any light that folks active on TCM and Accord could shed in terms of their>
best and worst-case scenarios on timelines for those features might help us>
narrow this down a bit. Otherwise, I'm inclined to defer to our past selves>
and fall back to "we agreed to yearly calendar releases for good reason.> Let's
stick to our guns.">> On Tue, Oct 24, 2023, at 9:37 AM, Brandon Williams
wrote:>> The concern I have with this is that is a big slippery 'if' that>
involves development time estimation, and if it tends to take longer> than we
estimate (as these things tend to do), then I can see a future> where 5.0 is
delayed until the middle of 2024, and I really don't want> that to happen.
Maybe it won't be a glamorous release but shipping> 5.0 mitigates our worst
case scenario.>> Kind Regards,> Brandon>> On Mon, Oct 23, 2023 at 4:02 PM
Dinesh Joshi <djo...@apache.org> wrote:> >> > I have a strong preference to
move out the 5.0 date to have accord and> TCM. I don’t see the point in
shipping 5.0 without these features> especially if 5.1 is going to follow close
behind it.> >> > Dinesh> >> > On Oct 23, 2023, at 4:52 AM, Mick Semb Wever
<m...@apache.org> wrote:> >> > > >> > The TCM work (CEP-21) is in its review
stage but being well past our> cut-off date¹ for merging, and now jeopardising
5.0 GA efforts, I would> like to propose the following.> >> > We merge TCM and
Accord only to trunk. Then branch cassandra-5.1 and> cut an immediate
5.1-alpha1 release.> >> > I see this as a win-win scenario for us, considering
our current> situation. (Though it is unfortunate that Accord is included in
this> scenario because we agreed it to be based upon TCM.)> >> > This will
mean…> > - We get to focus on getting 5.0 to beta and GA, which already has a>
ton of features users want.> > - We get an alpha release with TCM and Accord
into users hands quickly> for broader testing and feedback.> > - We isolate GA
efforts on TCM and Accord – giving oss and downstream> engineers time and
patience reviewing and testing. TCM will be the biggest> patch ever to land in
C*.> > - Give users a choice for a more incremental upgrade approach, given>
just how many new features we're putting on them in one year.> > - 5.1 w/ TCM
and Accord will maintain its upgrade compatibility with> all 4.x versions, just
as if it had landed in 5.0.> >> >> > The risks/costs this introduces are> > -
If we cannot stabilise TCM and/or Accord on the cassandra-5.1 branch,> and at
some point decide to undo this work, while we can throw away the> cassandra-5.1
branch we would need to do a bit of work reverting the> changes in trunk. This
is a _very_ edge case, as confidence levels on the> design and implementation
of both are already tested and high.> > - We will have to maintain an
additional branch. I propose that we> treat the 5.1 branch in the same
maintenance window as 5.0 (like we have> with 3.0 and 3.11). This also adds
the merge path overhead.> > - Reviewing of TCM and Accord will continue to
happen post-merge. This> is not our normal practice, but this work will have
already received its> two +1s from committers, and such ongoing review effort
is akin to GA> stabilisation work on release branches.> >> >> > I see no other
ok solution in front of us that gets us at least both the> 5.0 beta and
TCM+Accord alpha releases this year. Keeping in mind users> demand to start
experimenting with these features, and our Cassandra Summit> in December.> >>
>> > 1) https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3> >>
>>>>
