On Fri, Aug 16, 2013 at 11:15 AM, Fred Cooke <fred.co...@gmail.com> wrote:
> Dennis, I've been using (and mostly loving) the release plugin/process for
> the better part of a decade and certainly claim to understand it well. I
> don't see how my knowledge of that (or Sebb's perceived lack of knowledge
> of that) is in any way relevant. The release plugin means it's harder to do
> something wrong, and this is good, and why I love it. However, you can,
> could, and someone may, produce non-snapshot binaries on a staging server
> without having used it, from arbitrary sources and/or a bug in the release
> plugin could do something wrong, and given our collective faith in that
> plugin, you'd likely not check unless suspicious (eg when the assembly
> plugin suddenly stopped compressing some archives).
>
Yes I agree with all of that. That's why we need to check the content of a
source bundle during the release process.
That was not the point I was trying to make though. What I meant was that
if you understand what the Release plugin does, then you know where to look
for certain things, like the URL of the SCM tag.
>
> On Fri, Aug 16, 2013 at 8:32 PM, Dennis Lundberg <denn...@apache.org>
> wrote:
>
> > On Fri, Aug 16, 2013 at 9:52 AM, sebb <seb...@gmail.com> wrote:
> >
> > > On 16 August 2013 08:10, Dennis Lundberg <denn...@apache.org> wrote:
> > > > On Fri, Aug 16, 2013 at 1:20 AM, sebb <seb...@gmail.com> wrote:
> > > >
> > > >> On 15 August 2013 20:57, Dennis Lundberg <denn...@apache.org>
> wrote:
> > > >> > On Thu, Aug 15, 2013 at 9:27 PM, sebb <seb...@gmail.com> wrote:
> > > >> >
> > > >> >> On 15 August 2013 14:16, Jason van Zyl <ja...@tesla.io> wrote:
> > > >> >> > What Sebb is doing is perfectly reasonable.
> > > >> >>
> > > >> >
> > > >> > I agree. Checking that the source bundle is correct is good
> release
> > > >> review
> > > >> > practice.
> > > >> >
> > > >> > Thank you!
> > > >> >>
> > > >> >> > He's trying to assert that everything in the source ball
> actually
> > > >> comes
> > > >> >> from source control and that no errant files have made their way
> > into
> > > >> the
> > > >> >> distribution. Right now we cannot assert that the assembly plugin
> > has
> > > >> not
> > > >> >> wandered outside the checkout and pulled something else in, or
> that
> > > >> someone
> > > >> >> didn't accidentally put something else in the distribution. I
> think
> > > this
> > > >> >> unlikely but we can't assert otherwise right now which I believe
> is
> > > >> Sebb's
> > > >> >> point.
> > > >> >>
> > > >> >> It *has* already happened several times that I am aware of.
> > > >> >>
> > > >> >> The last few releases of the War plugin (various RMs & voters)
> > > >> >> included at least one spurious file.
> > > >> >> So it was not just a one-off packaging - and review - failure.
> > > >> >> [See separate thread(s) for all the details; they are not germane
> > > here.]
> > > >> >>
> > > >> >> The message is that mistakes happen even in automated processes.
> > > >> >> Which is why independent comparison of input and output is
> > valuable.
> > > >> >>
> > > >> >> > If we can embed the revision from which the assembly was made
> in
> > > the
> > > >> >> assembly itself (and maybe the build number plugin is doing this
> > > >> already),
> > > >> >> then a tool can be made to unpack the assembly, checkout the
> > revision
> > > >> and
> > > >> >> assert that everything in the source distribution comes from
> source
> > > >> control.
> > > >> >> >
> > > >> >> > If we can also assert that as part of each build all the
> license
> > > files
> > > >> >> are intact and headers are in place then I believe we're done
> with
> > > >> >> provenance.
> > > >> >> > Licenses are present, all files have valid license headers, all
> > > files
> > > >> >> present in the source distribution come from source control, all
> > > >> >> contributions to source control are from bonafide CLA carrying
> > Apache
> > > >> >> committers because you don't get access to commit until the CLA
> is
> > on
> > > >> file.
> > > >> >> >
> > > >> >> > Sebb, reasonably accurate?
> > > >> >>
> > > >> >> Yes.
> > > >> >>
> > > >> >> One other point I made already is that I think the vote e-mail
> > needs
> > > >> >> to be transparent to all, not just those on the PMC.
> > > >> >> Links to the output from the release process are obviously
> already
> > in
> > > >> >> the mail; what is missing is the input to the process, e.g.. SCM
> > > >> >> coords.
> > > >> >> Yes, it may be possible to dig out the details from the archive,
> > but
> > > >> >> that's not trivial.
> > > >> >>
> > > >> >
> > > >> > I disagree.
> > > >> >
> > > >> > If we focus first on a "normal" release, one that succeeds on the
> > > first
> > > >> > attempt, without a respin or deleting of tags.
> > > >> > To check provenance you would do this:
> > > >> >
> > > >> > 1. download the source bundle
> > > >> > 2. unpack the source bundle
> > > >> > 3. checkout the corresponding source code from the SCM
> > > >> > 4. compare the two trees
> > > >> >
> > > >> > Right so far?
> > > >> >
> > > >> > What you want, if I understand you correctly, is to have the SCM
> URL
> > > in
> > > >> the
> > > >> > vote email. So that you can give that to your SCM client in step
> 3.
> > > >>
> > > >> Yes.
> > > >>
> > > >> > With the process we use here at the Maven project, the SCM URL is
> in
> > > the
> > > >> > pom.xml file that sits in the root directory of the unpacked
> source
> > > >> bundle.
> > > >> > All you need to do is open the file and copy the URL from there. I
> > > still
> > > >> > fail to see how that is so much harder than to copy the URL from
> an
> > > >> email.
> > > >> >
> > > >> > That is if you don't know the conventions that we use, by way of
> the
> > > >> > Release Plugin. The tag will always be in the format
> > > >> > ${project.artifactId}-${project.version}
> > > >> >
> > > >>
> > > >> My point is that it should be completely transparent, even to
> outside
> > > >> reviewers.
> > > >>
> > > >
> > > > I guess that this is the point that we'll have to agree to disagree
> on.
> > > My
> > > > view is that if someone wants to to review a release from the Maven
> > > > project, they'd have to have a basic understanding of how Maven works
> > and
> > > > how we do releases in the Maven project. That includes what the
> Release
> > > > Plugin is and how it works.
> > >
> > > Why does the release tool matter?
> > >
> >
> > It is not the tool itself that matters, but rather what the tools does.
> > Since the Release plugin automates a lot of the tasks involved when
> doing a
> > release, it can appear to make things less transparent. Therefor a
> > knowledge about what it does is needed to make it transparent.
> >
> > It's still an ASF release.
> > >
> >
> > Yes, and we must adhere to all the requirements of an ASF release. There
> > are however a lot of different ways to make an ASF compliant release.
> Each
> > PMC decides on a process for that, that works best for them.
> >
> > It should not matter how the source release artifacts are generated.
> > > The process of checking them is still the same.
> > >
> > > Of course if things go wrong, then it's important to know how the SCM
> > > input was assembled into the source archives in order to fix the
> > > process.
> > > But the release tool is completely irrelevant to the process of
> > > checking the contents of the source archives.
> > >
> >
> > See above.
> >
> >
> > >
> > > > Most people have their own checklist of stuff they do when reviewing
> a
> > > > release. Would it be a good idea if we aggregate all those points on
> a
> > > page
> > > > on the Maven web site, under the development section? That would also
> > > serve
> > > > as a guide for "outside reviewers".
> > >
> > > Yes, though that is a separate issue.
> > >
> >
> > Agreed.
> >
> > >
> > > >> > Now, for a respun release thing are trickier. Here it might be a
> > good
> > > >> idea
> > > >> > to include the revision number or hash, or whatever is unique in
> the
> > > SCM
> > > >> > being used.
> > > >>
> > > >> And how do you know from a vote e-mail that it is respun?
> > > >>
> > > >
> > > > It should always say so in the subject of the vote email. Not sure if
> > > that
> > > > is written down somewhere, but that's how we have always done it. If
> > it's
> > > > missing I'll add it to our release process document.
> > > >
> > > >
> > > >>
> > > >> > Even though the code under review will always be under the
> > > >> > latest tag in the above format (at least for Subversion).
> > > >>
> > > >> Until the next respin.
> > > >>
> > > >> If there is a respin, and reviewers are not following the e-mails
> very
> > > >> carefully, it would be quite easy to overlook an updated tag.
> > > >>
> > > >> This is all about making sure that it is really obvious what the
> vote
> > is
> > > >> about.
> > > >>
> > > >
> > > > Yes, that's why I agreed that it's a good idea to add the
> revision/hash
> > > > when doing a respin.
> > > >
> > > >
> > > >>
> > > >> >
> > > >> >> Publishing the SCM coordinates in the mail is trivial to do, and
> > > shows
> > > >> >> that the input is an important part of the review process.
> > > >> >> Having the information in the mail thread is also useful for the
> > > >> archives.
> > > >> >>
> > > >> >
> > > >> > As others have said before, we aim to automate the release process
> > as
> > > >> much
> > > >> > as possible. Therefor even a seemingly minor addition will be
> > > questioned,
> > > >> > as you have noticed, before it is included in our process.
> > > >> >
> > > >> > Can you explain why the information is useful for the archives?
> I've
> > > seen
> > > >> > you mentioned that before. Isn't that moot once the release is
> > > approved?
> > > >> > The tag will be in Subversion for the forseable future and noone
> > will
> > > >> touch
> > > >> > it. What am I missing?
> > > >>
> > > >> Why would a release need to be revisited?
> > > >> Perhaps someone is complaining that one of our releases contains
> code
> > > >> it should not.
> > > >> If that is the case, it helps to have the evidence of the release
> vote
> > > >> in plain sight.
> > > >>
> > > >
> > > > Sure it helps in those cases. But the evidence in our case is in the
> > the
> > > > source bundle itself, which is even better in my opinion. There you
> > have
> > > > the tag in the POM file.
> > >
> > > However, the link to the staging repo is temporary.
> > > Once the repo is published, the staging repo is deleted and there is
> > > no direct route to the pom from the vote e-mail.
> > >
> >
> > Once the staging repo is promoted, its content is moved to the real
> > repository. So the source bundle is available there for anyone who wants
> to
> > track down the complete contents of a previous release.
> >
> > Again this is where a bit of knowledge of what the Release Plugin does
> > helps you. A release done with the Release Plugin will always be
> available
> > at
> >
> >
> https://repository.apache.org/content/repositories/releases/${project.groupId}/${project.artifactId}/${project.version}/${project.artifactId}-${project.version}-source-release.zip
> > where project.groupId has had '.' replaced by '/'.
> >
> >
> > > > Note that I'm not talking about respins here, that's another story.
> > >
> > > >
> > > >>
> > > >> >
> > > >> >>
> > > >> >> > On Aug 15, 2013, at 9:01 AM, Chris Graham <
> chrisgw...@gmail.com>
> > > >> wrote:
> > > >> >> >
> > > >> >> >>
> > > >> >> >>
> > > >> >> >> Sent from my iPhone
> > > >> >> >>
> > > >> >> >> On 15/08/2013, at 10:05 PM, sebb <seb...@gmail.com> wrote:
> > > >> >> >>
> > > >> >> >>> On 15 August 2013 10:08, Chris Graham <chrisgw...@gmail.com>
> > > wrote:
> > > >> >> >>>> What sebb does not appear to have understood or accepted, as
> > > >> Stephen
> > > >> >> has
> > > >> >> >>>> endlessly pointed out, is that we vote on the source bundle,
> > > not a
> > > >> scm
> > > >> >> >>>> revision, and that, strictly speaking a SCM is not even
> > required
> > > >> >> (however
> > > >> >> >>>> sensible it is to use one).
> > > >> >> >>>>
> > > >> >> >>>> He wants a tree and a revision so that we can compare
> between
> > > >> >> releases,
> > > >> >> >>>> where what he should be doing, strictly speaking, is
> comparing
> > > >> source
> > > >> >> tar
> > > >> >> >>>> balls, as that is what we really are voting on.
> > > >> >> >>>
> > > >> >> >>> I agree that what is released (and voted on) are the source
> > > >> tarballs.
> > > >> >> >>> And any such tarballs should be identical (barring possibly
> > > >> different
> > > >> >> >>> EOL settings for text files).
> > > >> >> >>>
> > > >> >> >>> However, that is only one of the checks that need to be made.
> > > >> >> >>>
> > > >> >> >>> The PMC also needs to ensure that the files are being
> released
> > > under
> > > >> >> >>> the correct license.
> > > >> >> >>
> > > >> >> >> Are not the licenses in the source that is in the source
> > tarball?
> > > >> >> >>
> > > >> >> >> If so, can not the rat plugin or similar be used to check the
> > > >> >> compliance?
> > > >> >> >>
> > > >> >> >>> I contend that the only practical way to check the licences
> is
> > to
> > > >> >> >>> compare the source tarball(s) with the files in SCM.
> > > >> >> >>>
> > > >> >> >>> [The files should only be added to SCM if the license is OK,
> so
> > > the
> > > >> >> >>> SCM tag acts as a database of validated source files.]
> > > >> >> >>>
> > > >> >> >>> The SVN revision / Git hash are needed to ensure uniqueness.
> > > >> >> >>>
> > > >> >> >>>
> > > >>
> ---------------------------------------------------------------------
> > > >> >> >>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > >> >> >>> For additional commands, e-mail: dev-h...@maven.apache.org
> > > >> >> >>>
> > > >> >> >>
> > > >> >> >>
> > > ---------------------------------------------------------------------
> > > >> >> >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > >> >> >> For additional commands, e-mail: dev-h...@maven.apache.org
> > > >> >> >>
> > > >> >> >
> > > >> >> > Thanks,
> > > >> >> >
> > > >> >> > Jason
> > > >> >> >
> > > >> >> > ----------------------------------------------------------
> > > >> >> > Jason van Zyl
> > > >> >> > Founder, Apache Maven
> > > >> >> > http://twitter.com/jvanzyl
> > > >> >> > ---------------------------------------------------------
> > > >> >> >
> > > >> >> > There's no sense in being precise when you don't even know what
> > > you're
> > > >> >> talking about.
> > > >> >> >
> > > >> >> > -- John von Neumann
> > > >> >> >
> > > >> >> >
> > > >> >> >
> > > >> >> >
> > > >> >> >
> > > >> >> >
> > > >> >>
> > > >> >>
> > ---------------------------------------------------------------------
> > > >> >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > >> >> For additional commands, e-mail: dev-h...@maven.apache.org
> > > >> >>
> > > >> >> --
> > > >> >> Dennis Lundberg <dev-h...@maven.apache.org>
> > > >> >>
> > > >>
> > > >>
> ---------------------------------------------------------------------
> > > >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > >> For additional commands, e-mail: dev-h...@maven.apache.org
> > > >>
> > > >> --
> > > >> Dennis Lundberg <dev-h...@maven.apache.org>
> > > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > For additional commands, e-mail: dev-h...@maven.apache.org
> > >
> > > --
> > > Dennis Lundberg <dev-h...@maven.apache.org>
> > >
> >
>
> --
> Dennis Lundberg
