>
> So who and when is going to implement the automation?
I don't believe we have sufficient consensus that this is necessary to
start doling out action-items for implementation.
On Thu, May 7, 2015 at 10:16 AM, Ariel Weisberg <ariel.weisb...@datastax.com
> wrote:
> Hi,
>
> If it were automated I would have no problem with it. That would be less
> work for me because the problems detected would occur anyways and have to
> be dealt with by me. I just don't want to deal with extra steps and latency
> manually.
>
> So who and when is going to implement the automation?
>
> Ariel
>
> On Thu, May 7, 2015 at 11:11 AM, Benedict Elliott Smith <
> belliottsm...@datastax.com> wrote:
>
> > It's odd, because I honestly think this release process will be easier,
> > since the stricter we make it the smoother it can become. It requires
> well
> > formed commits from everyone, and lets the committers asynchronously
> > confirm their work, and for it to never be in question *who* needs to fix
> > something, nor what the effect of their fixing it will be. It means we
> can,
> > as Ariel said, perform a bisect and honestly know its result is accurate.
> > Small commits don't need to worry about fast-forwarding; in fact, nobody
> > does. It can either be automated, or we can fast forward at a time that
> > suits us. In which case the process is *the same* as it is currently.
> >
> > I have no interest in making the commit process harder.
> >
> >
> > On Thu, May 7, 2015 at 3:59 PM, Jake Luciani <jak...@gmail.com> wrote:
> >
> > > Ok let's focus then on the idea that trunk is releasable. Releasable
> > > to me doesn't mean it can't contain a bad merge.
> > >
> > > It means it doesn't contain some untested and unstable feature. We
> > > can always "release from trunk" and we still have a release process.
> > >
> > > The idea that trunk must contain. a first time it hits the branch,
> > > releasable code is way overboard
> > >
> > > On Thu, May 7, 2015 at 10:50 AM, Benedict Elliott Smith
> > > <belliottsm...@datastax.com> wrote:
> > > >>
> > > >> This breaks your model of applying every commit ref by ref.
> > > >
> > > >
> > > > How? The rebase only affects commits after the "real" branch, so it
> > still
> > > > cleanly fast forwards?
> > > >
> > > > Merging is *hard*. Especially 2.1 -> 3.0, with many breaking API
> > changes
> > > > (this is before 8099, which is going to make a *world* of hurt, and
> > will
> > > > stick around for a year). It is *very* easy to break things, with
> even
> > > the
> > > > utmost care.
> > > >
> > > > On Thu, May 7, 2015 at 3:46 PM, Jake Luciani <jak...@gmail.com>
> wrote:
> > > >
> > > >> You then fetch and repair
> > > >> your local version and try again.
> > > >>
> > > >> This breaks your model of applying every commit ref by ref.
> > > >>
> > > >> I'm all for trying to avoid extra work/stability but we already have
> > > >> added a layer of testing every change before commit. I'm not going
> to
> > > >> accept we need to also add a layer of testing before every merge.
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> On Thu, May 7, 2015 at 10:36 AM, Benedict Elliott Smith
> > > >> <belliottsm...@datastax.com> wrote:
> > > >> >>
> > > >> >> wouldn't you need to force push?
> > > >> >
> > > >> >
> > > >> > git push --force-with-lease
> > > >> >
> > > >> > This works essentially like CAS; if the remote repositories are
> not
> > > the
> > > >> > same as the one you have modified, it will fail. You then fetch
> and
> > > >> repair
> > > >> > your local version and try again.
> > > >> >
> > > >> > So what does this buy us?
> > > >> >
> > > >> >
> > > >> > This buys us a clean development process. We bought into "always
> > > >> > releasable". It's already a tall order; if we start weakening the
> > > >> > constraints before we even get started, I am unconvinced we will
> > > >> > successfully deliver. A monthly release cycle requires *strict*
> > > >> processes,
> > > >> > not *almost* strict, or strict*ish*.
> > > >> >
> > > >> > Something that could also help make a more streamlined process: if
> > > actual
> > > >> > commits were constructed on development branches ready for commit,
> > > with a
> > > >> > proper commit message and CHANGES.txt updated. Even more ideally:
> > with
> > > >> git
> > > >> > rerere data for merging up to each of the branches. If we had
> that,
> > > and
> > > >> > each of the branches had been tested in CI, we would be much
> closer
> > > than
> > > >> we
> > > >> > are currently, as the risk-at-commit is minimized.
> > > >> >
> > > >> > On Thu, May 7, 2015 at 2:48 PM, Jake Luciani <jak...@gmail.com>
> > > wrote:
> > > >> >
> > > >> >> git rebase -i trunk_staging <ref~1>
> > > >> >> fix the problem
> > > >> >> git rebase --continue
> > > >> >>
> > > >> >> In this situation, if there was an untested follow on commit
> > wouldn't
> > > >> >> you need to force push?
> > > >> >>
> > > >> >> On Thu, May 7, 2015 at 9:28 AM, Benedict Elliott Smith
> > > >> >> <belliottsm...@datastax.com> wrote:
> > > >> >> >>
> > > >> >> >> If we do it, we'll end up in weird situations which will be
> > > annoying
> > > >> for
> > > >> >> >> everyone
> > > >> >> >
> > > >> >> >
> > > >> >> > Such as? I'm not disputing, but if we're to assess the relative
> > > >> >> > strengths/weaknesses, we need to have specifics to discuss.
> > > >> >> >
> > > >> >> > If we do go with this suggestion, we will most likely want to
> > > enable a
> > > >> >> > shared git rerere cache, so that rebasing is not painful when
> > there
> > > >> are
> > > >> >> > future commits.
> > > >> >> >
> > > >> >> > If instead we go with "repairing" commits, we cannot have a
> > > "queue" of
> > > >> >> > things to merge up to. Say you have a string of commits waiting
> > for
> > > >> >> > approval C1 to C4; you made C1, and it broke something. You
> > > introduce
> > > >> C5
> > > >> >> to
> > > >> >> > fix it, but the tests are still broken. Did you not really fix
> > it?
> > > Or
> > > >> >> > perhaps one of C2 to C4 are to blame, but which? And have you
> > > >> >> accidentally
> > > >> >> > broken *them* with your commit? Who knows. Either way, we
> > > definitely
> > > >> >> cannot
> > > >> >> > fast forward. At the very best we can hope that the new merge
> did
> > > not
> > > >> >> > conflict or mess up the other people's C2 to C4 commits, and
> they
> > > >> have to
> > > >> >> > now merge on top. But what if another merge comes in, C6, in
> the
> > > >> >> meantime;
> > > >> >> > and C2 really did also break the tests in some way; how do we
> > > >> determine
> > > >> >> C2
> > > >> >> > was to blame, and not C6, or C3 or C4? What do the committers
> for
> > > >> each of
> > > >> >> > these do? We end up in a lengthy tussle, and aren't able to
> > commit
> > > >> any of
> > > >> >> > these to the mainline until all of them are resolved. Really we
> > > have
> > > >> to
> > > >> >> > prevent any merges to the staging repository until the mistakes
> > are
> > > >> >> fixed.
> > > >> >> > Since our races in these scenario are the length of time taken
> > for
> > > >> cassci
> > > >> >> > to vet them, these problems are much more likely than current
> > race
> > > to
> > > >> >> > commit.
> > > >> >> >
> > > >> >> > In the scheme I propose, in this scenario, the person who broke
> > the
> > > >> build
> > > >> >> > rebases everyone's branches to his now fixed commit, and the
> next
> > > >> broken
> > > >> >> > commit gets blamed, and all other commits being merged in on
> top
> > > can
> > > >> go
> > > >> >> in
> > > >> >> > smoothly. The only pain point I can think of is the
> multi-branch
> > > >> rebase,
> > > >> >> > but this is solved by git rerere.
> > > >> >> >
> > > >> >> > I agree running tests is painful, but at least for the build,
> > this
> > > >> should
> > > >> >> >> be the responsibility of the committer to build before merging
> > > >> >> >
> > > >> >> >
> > > >> >> > Why make the distinction if we're going to have staging
> commits?
> > > It's
> > > >> a
> > > >> >> bit
> > > >> >> > of a waste of time to run three ant real-clean && ant tasks,
> and
> > > >> >> increases
> > > >> >> > the race window for merging (which is painful whether or not
> > > involves
> > > >> a
> > > >> >> > rebase), and it is not a *typical* occurrence ("alarming" is
> > > >> subjective)
> > > >> >> >
> > > >> >> > On Thu, May 7, 2015 at 2:12 PM, Sylvain Lebresne <
> > > >> sylv...@datastax.com>
> > > >> >> > wrote:
> > > >> >> >
> > > >> >> >> > If one of them breaks, we go and edit the _staging branch in
> > > place
> > > >> to
> > > >> >> >> correct
> > > >> >> >> > the problem, and let CI run again.
> > > >> >> >>
> > > >> >> >> I would strongly advise against *in place* edits. If we do it,
> > > we'll
> > > >> >> end up
> > > >> >> >> in
> > > >> >> >> weird situations which will be annoying for everyone. Editing
> > > commits
> > > >> >> that
> > > >> >> >> have
> > > >> >> >> been shared is almost always a bad idea and that's especially
> > true
> > > >> for
> > > >> >> >> branch
> > > >> >> >> that will have some amount of concurrency like those staging
> > > >> branches.
> > > >> >> >>
> > > >> >> >> Even if such problems are rare, better to avoid them in the
> > first
> > > >> place
> > > >> >> by
> > > >> >> >> simply
> > > >> >> >> commit new "fixup" commits as we currently do. Granted this
> give
> > > you
> > > >> a
> > > >> >> >> slightly
> > > >> >> >> less clean history but to the best of my knowledge, this
> hasn't
> > > been
> > > >> a
> > > >> >> pain
> > > >> >> >> point so far.
> > > >> >> >>
> > > >> >> >> > wait for CI; all clear
> > > >> >> >> >
> > > >> >> >> > git checkout cassandra-2.0; git merge cassandra-2.0_staging
> > > >> >> >> > git checkout cassandra-2.1; git merge cassandra-2.1_staging
> > > >> >> >> > git checkout trunk; git merge trunk_staging
> > > >> >> >> >
> > > >> >> >> > This does introduce some extra steps to the merge process
> > > >> >> >>
> > > >> >> >> If we do this, we should really automate that last part (have
> > the
> > > CI
> > > >> >> >> environment merge the staging branch to the non-staging ones
> on
> > > >> >> success).
> > > >> >> >>
> > > >> >> >> > It seems if we want an "always releasable" set of branches,
> we
> > > need
> > > >> >> >> something
> > > >> >> >> > along these lines.
> > > >> >> >>
> > > >> >> >> Agreed as far as having staging branches vetoed by CI goes.
> Less
> > > sure
> > > >> >> about
> > > >> >> >> the edit-commit-in-place part as said above.
> > > >> >> >>
> > > >> >> >> > I certainly break tests by mistake, or the build itself,
> with
> > > >> alarming
> > > >> >> >> regularity.
> > > >> >> >>
> > > >> >> >> I agree running tests is painful, but at least for the build,
> > this
> > > >> >> should
> > > >> >> >> be
> > > >> >> >> the responsibility of the committer to build before merging.
> We
> > > all
> > > >> >> forget
> > > >> >> >> it from
> > > >> >> >> times to times and that's ok, but it's not ok if it's
> > "alarmingly
> > > >> >> regular".
> > > >> >> >>
> > > >> >> >> --
> > > >> >> >> Sylvain
> > > >> >> >>
> > > >> >>
> > > >> >>
> > > >> >>
> > > >> >> --
> > > >> >> http://twitter.com/tjake
> > > >> >>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> http://twitter.com/tjake
> > > >>
> > >
> > >
> > >
> > > --
> > > http://twitter.com/tjake
> > >
> >
>
--
Joshua McKenzie
DataStax -- The Apache Cassandra Company
