I agree, decoupling seems better to me. The fewer inter-effort dependencies, the less foot-treading.
As Henrik mentioned, we don’t need a feature complete cep-15 to merge, as its integration is quite lightweight. We can leave some stuff to provide later, and depending on conditions at release time make a decision as to whether we release with cep-21, or with a minimal epoch facility (which would IMO not being a huge endeavour), or with it disabled altogether.
On 24 Mar 2023, at 19:13, Josh McKenzie <jmcken...@apache.org> wrote:
FWIW, I'd still rather just integrate w/ TCM ASAP, avoiding integration risk while accepting the possible delivery risk.
What does the chain of rebases against trunk look like here? cep-21-tcm rebase, then cep-15 on cep-21-tcm, then cep-7 on cep-21-tcm, then a race on whichever of 15 or 7 merge after 21 goes into trunk? Or 7 based on 15, the other way around...
I'm not actively working on any of these branches so take my perspective with that appropriate grain of salt, but the coupling of these things seems to have it's own kind of breed of integration pain to upkeep over time depending on how frequently we're rebasing against trunk.
the question we want to answer is whether or not we build a throwaway patch for linearizable epochs
Do we have an informed opinion on how long we think this would take? Seems like that'd help clarify whether or not there's contributors with the bandwidth and desire to even do that or whether everyone depending on cep-21 is our option.
On Fri, Mar 24, 2023, at 1:30 PM, Caleb Rackliffe wrote:
It wasn't too terrible, and I was actually able to get the main CQL-based Accord tests working as long as I disabled automatic forwarding of CAS and SERIAL read operations to Accord. The bigger issue was test stability in cep-21-tcm. I'm sure that will mature quickly here, and I created a few issues to attach to the Transactional Metadata epic.
In the meantime, I rebased cep-15-accord on trunk at commit 3eb605b4db0fa6b1ab67b85724a9cfbf00aae7de. The option to finish the remaining bits of CASSANDRA-18196 and merge w/o TCM is still available, but it sounds like the question we want to answer is whether or not we build a throwaway patch for linearizable epochs in lieu of TCM?
FWIW, I'd still rather just integrate w/ TCM ASAP, avoiding integration risk while accepting the possible delivery risk.
making sure that joining and leaving nodes update some state via Paxos instead of via gossip
What kind of a time delivery risk does coupling CEP-15 with CEP-21 introduce (i.e. unk-unk on CEP-21 leading to delay cascades to CEP-15)? Seems like having a table we CAS state for on epochs wouldn't be too challenging, but I'm not familiar w/the details so I could be completely off here.
Being able to deliver both of these things on their own timetable seems like a pretty valuable thing assuming the lift required would be modest.
On Fri, Mar 24, 2023, at 6:15 AM, Benedict wrote:
Accord doesn’t have a hard dependency on CEP-21 fwiw, it just needs linearizable epochs. This could be achieved with a much more modest patch, essentially avoiding almost all of the insertion points of cep-21, just making sure that joining and leaving nodes update some state via Paxos instead of via gossip, and assign an epoch as part of the update.
It would be preferable to use cep-21 since it introduces this functionality, and our intention is to use cep-21 for this. But it isn’t a hard dependency.
Since Accord depends on transactional meta-data... is there really any alternative than what you propose?
Sure, if there is some subset of Accord that could be merged, while work continues on a branched that is based on CEP-21 branch, that would be great. Merging even a prototype of Accord to trunk probably has marketing value. (Don't laugh, many popular databases have had "atomic transactions, except if anyone executes DDL simultaneously".)
We've already talked a bit about how and when the current Accord feature branch should merge to trunk. Earlier today, the cep-21-tcm branch was created to house ongoing work on Transactional Metadata.
Returning to CASSANDRA-18196 (merging Accord to trunk) after working on some other issues, I want to propose changing direction slightly, and make sure this makes sense to everyone else.
1.) There are a few minor Accord test issues in progress that I'd like to wrap up before doing anything, but those shouldn't take long. (See CASSANDRA-18302 and CASSANDRA-18320.) 2.) Accord logically depends on Transactional Metadata.
3.) The new cep-21-tcm branch is going to have to be rebased to trunk on a regular basis.
So...
4.) I would like to pause merging cep-15-accord to trunk, and instead rebase it on cep-21-tcm until such time as the latter merges to trunk, at which point cep-15-accord can be rebased to trunk again and then merged when ready, nominally taking up the work of CASSANDRA-18196 again.
Any objections to this?
--
|
