I am certainly not proposing any certainty about outside interest, but I think as the only full implementation of a leaderless protocol in existence, as well as an open source pluggable distributed transaction protocol, the chance of some interest is not vanishingly small (once it is proven in Cassandra). Whether that will translate to any useful interest is even less certain, but I think it would be valuable if it transpires.
It’s also not the only reason to develop this project as a library, and wasn’t one of the reasons given when we discussed this when the proposal was discussed initially. The main reasons were ensuring that the transaction system in Cassandra was not tied to Accord, and to allow better testing in isolation.
Regarding the use of snapshots, this isn’t impossible Henrik - I floated this as an option. But besides the additional overhead during development, this does not maintain reproducible builds, as the snapshot can change. Perhaps we could introduce permanent snapshots for a given SHA, but at that point it seems to just make more sense to use submodules. Including snapshots inside the lib directory until release would seem to be fine, and then perform parallel release votes.
On 17 Jan 2023, at 21:04, Henrik Ingo <henrik.i...@datastax.com> wrote:
Hi Derek
Somewhat of a newcomer myself, it seems the answers to your excellent questions are:
* We don't all agree with the premise that Accord will attract substantial outside interest. Even so, my personal opinion is that whether that happens or not, it's not wrong to aspire toward or plan for such a future.
* Yes, just using Accord as a library dependency would be the normal thing to do, but that introduces a need to create Accord releases to match Cassandra releases. Since ASF mandates a 3 day voting process to release software artifacts, this creates a lot of bureaucratic overhead, which is why this otherwise sane alternative is nobody's favorite. (Cassandra releases cannot or should not depend on snapshot releases of libraries.
* So we are discussing various alternatives that keep Accord separate, while at the same time recording some link about which exact version of Accord was checked out.
henrik Actually, re-reading the thread, I think I missed the initial point
brought up and got lost in the discussion specific to submodules. What
is the technical reason for bringing Accord in-tree? While I think
submodules are the best way to include source in-tree, I'm not sure
this is actually the correct thing to do in this case. Don't we
already have mechanisms to deal with snapshot versions of library
dependencies in the build? Do we need release votes for snapshots?
Thanks,
Derek
On Tue, Jan 17, 2023 at 9:25 AM Derek Chen-Becker <de...@chen-becker.org> wrote:
>
> I'd like to go back to Benedict's initial point: if we have a new
> consensus protocol that other projects would potentially be interested
> in, then by all means it should be its own project. Let's start with
> that as a basis for discussion, because from my reading it seems like
> people might be disagreeing with that initial premise.
>
> If we agree that Accord should be independent, I'm +1 for git
> submodules primarily because that's a standard way of doing things and
> I don't think we need yet another bespoke solution to a problem that
> hundreds, if not thousands of other software projects encounter. I've
> worked with lots of projects using submodules and while they're not a
> panacea, they've never been a significant problem to work with.
>
> It's also a little confusing to see people argue about HEAD in
> relation to any of this, since that's just an alias to the latest
> commit for a given branch. In every project I've worked with that uses
> submodules you would never use HEAD, because the submodule itself
> already records the *exact* commit associated with the parent.
>
> Cheers,
>
> Derek
>
> On Tue, Jan 17, 2023 at 2:28 AM Benedict <bened...@apache.org> wrote:
> >
> > The answer to all your questions is “like any other library” - this is a procedural hack to ease development. There are alternative isomorphic hacks, like compiling source jars from Accord and including them in the C* tree, if it helps your mental model.
> >
> > > you stated that a goal was to avoid maintaining multiple branches.
> >
> > No, I stated that a goal was to *decouple* development of Accord from C*. I don’t see why you would take that to mean there are no branches of Accord, as that would quite clearly be incompatible with the C* release strategy.
> >
> >
> >
> > On 17 Jan 2023, at 07:36, Mick Semb Wever <m...@apache.org> wrote:
> >
> >
> >>
> >> … extrapolating this experience to multiple C* versions
> >
> >
> > To include forward-merging, bisecting old history, etc etc. that's a leap of faith that I believe deserves the discussion.
> >
> >> - patches are off submodule SHAs, not the submodule's HEAD,
> >>
> >>
> >> A SHA would point to the HEAD of a given branch, at the time of merge, just by SHA? I’ve no idea what you imagine here, but this just ensures that a given SHA of the importing project continues to compile correctly when it is no longer HEAD. It does not mean there’s no HEAD that corresponds directly to the SHA of the importing project’s HEAD.
> >
> >
> >
> > That wasn't my concern. Rather that you need to know in advance when the SHA is not HEAD. You can't commit off a past SHA. Once you find out (and how does this happen?) that the submodule code is not HEAD what do you then do? What if fast-forwarding the submodule to HEAD's SHA breaks things, do you now have to fix that or introduce branching in the submodule? If the submodule doesn't have releases, is it doing versioning, and if not how are branches distinguished?
> >
> > Arn't these all fair enquiries to raise?
> >
> >> - you need to be making commits to all branches (and forward merging) anyway to update submodule SHAs,
> >>
> >>
> >> Exactly as you would any library upgrade?
> >
> >
> >
> > Correct. submodules does not solve/remove the need to commit to multiple branches and forward merge.
> > Furthermore submodules means at least one additional commit, and possibly twice as many commits.
> >
> >
> >> - if development is active on trunk, and then you need an update on an older branch, you have to accommodate to backporting all those trunk changes (or introduce the same branching in the submodule),
> >>
> >>
> >> If you do feature development against Accord then you will obviously branch it? You would only make bug fixes to a bug fix branch. I’m not sure what you think is wrong here.
> >
> >
> >
> > That's not obvious, you stated that a goal was to avoid maintaining multiple branches. Sure there's benefits to a lazy branching approach, but it contradicts your initial motivations and introduces methodology changes that are worth pointing out. What happens when there are multiple consumers of Accord, and (like the situation we face with jamm) its HEAD is well in front of anything C* is using.
> >
> > As Henrik states, the underlying problem doesn't change, we're just choosing between trade-offs. My concern is that we're not even doing a very good job of choosing between the trade-offs. Based on past experiences with submodules: that started with great excitement and led to tears and frustration after a few years; I'm only pushing for a more thorough discussion and proposal.
> >
> >
> >
> >
>
>
> --
> +---------------------------------------------------------------+
> | Derek Chen-Becker |
> | GPG Key available at https://urldefense.com/v3/__https://keybase.io/dchenbecker__;!!PbtH5S7Ebw!elHEL4NgggbLsBElKkANZ1KhFuuOpnWrUZe8RESgdwNEkPdmWuw_JQEVFlxZbPTkq7FsCt5BAn_x5pBGaqIJig$ and |
> | https://urldefense.com/v3/__https://pgp.mit.edu/pks/lookup?search=derek*40chen-becker.org__;JQ!!PbtH5S7Ebw!elHEL4NgggbLsBElKkANZ1KhFuuOpnWrUZe8RESgdwNEkPdmWuw_JQEVFlxZbPTkq7FsCt5BAn_x5pBWlY_cFw$ |
> | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7 7F42 AFC5 AFEE 96E4 6ACC |
> +---------------------------------------------------------------+
--
+---------------------------------------------------------------+
| Derek Chen-Becker |
| GPG Key available at https://urldefense.com/v3/__https://keybase.io/dchenbecker__;!!PbtH5S7Ebw!elHEL4NgggbLsBElKkANZ1KhFuuOpnWrUZe8RESgdwNEkPdmWuw_JQEVFlxZbPTkq7FsCt5BAn_x5pBGaqIJig$ and |
| https://urldefense.com/v3/__https://pgp.mit.edu/pks/lookup?search=derek*40chen-becker.org__;JQ!!PbtH5S7Ebw!elHEL4NgggbLsBElKkANZ1KhFuuOpnWrUZe8RESgdwNEkPdmWuw_JQEVFlxZbPTkq7FsCt5BAn_x5pBWlY_cFw$ |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7 7F42 AFC5 AFEE 96E4 6ACC |
+---------------------------------------------------------------+
--
|
