Hmm. I didn't think there were "sides".
The fact that Apache Royale has 100 steps has nothing to do with using a CI
server to create release artifacts. It is just because Royale not only
distributes maven artifacts from 3 repos, but also two flavors of legacy
artifacts are created and distributed via dist.a.o and NPM as well. Our RMs
have 100 steps even if they use a local machine. The Maven artifacts are
created in the first 20-ish commands.
I certainly hope I didn't claim that Royale's process is perfect and a
reference model. I've been trying to say that it now seems practical to use a
CI server in the release process because there are reproducible binary plugins
for Maven. Royale is in fact doing that, but because reproducible binaries are
relatively new, I would not say that what we have is a reference model. We
just have something that works.
To provide a bit more detail on what we've got working: where an RM on a local
machine typically does a release:branch, release:prepare, and release:perform,
on a CI server, I think there needs to be more work between each step. So you
can't have a single CI job that runs all three steps, you have to have a CI job
for each step, and do some work between steps and manually launch the next step.
That's because the CI server has to run release:branch with pushChanges=false
because there shouldn't be any committer credentials on the CI server. The RM
has to login and push the changes by supplying username and password. Then the
CI server can run release:prepare, also with pushChanges=false. And again, the
RM has to login and push changes and the tag.
Then when the CI server runs release:perform, the flag skipGPG=true has to be
set. And we also add an altDeploymentDirectory. That's because
release:perform not only generates the binaries, but it signs them and uploads
them to the staging server. The CI server has to skip signing since the keys
can't be stored on the server. So what our CI job does is redirect the binary
artifacts to an altDeploymentDirectory and use Jenkins to archive the
altDeploymentDIrectory as build artifacts. Then the RM downloads the artifacts
to a local machine, gets the source package and builds it, then compares the
jars that were built with the ones in the altDeploymentDIrectory. If they
match, the RM can sign the artifacts and then use Maven's Wagon to upload them
to the staging server.
Ideas for improvement are welcome. This is just the first thing that came to
mind that worked and seemed secure and conformed to Apache's rules/policies.
As you can see the RM still has to build the source code locally, all voters
need to be able to build the source anyway, and the RM can theoretically save
some time by skipping some/all tests because they were run on the CI server and
the binaries are bit-for-bit identical.
HTH,
-Alex
On 4/14/20, 11:59 PM, "Christofer Dutz" <christofer.d...@c-ware.de> wrote:
Hi Gabriel,
I just have to jump in here ... as I saw you were only getting feedback
from one side.
I did see that Cloudstack generally seems to be built with Maven, so I can
give some input
There is absolutely nothing wrong with building dev snapshots on a CI
server ... that's one of the things they were built for.
However when talking about releases the situation is different. Because you
need a real persons credentials for:
- Pushing to Git repos
- Signing artifacts
- Deploying artifact to any form of remote repos
So these are the parts you can't automate and a RM will have to jump in
manually.
So what's typically a 3 command release process on an ordinary maven
project:
- branch
- prepare
- release
Has to be split up and divided into a sequence of steps where some can be
done "automatic" (by clicking a button) and some have to be done manually.
I would also strongly suggest not to use the Apache Royale release process
as a reference. It was intended to simplify things and currently is a process
of about 100 steps that have to be executed in sequence ... some by clicking a
button ... some by logging in on the CI server (which would probably be a
problem on shared ASF build nodes).
I doubt this is something you would like to do. I. strongly suggest not to.
So please keep in mind that you can't have any non-technical user
credentials on a shared CI server node ... you could probably do that if you
have a dedicated Release Manager machine (Sort of you start it up, provide the
credentials needed and after the Release the VM is discarded) but it's totally
out of the question for a shared machine. And as Apache doesn't have technical
users outside of infra, well I guess the only options you have are:
- dedicated RM machine
- very lengthy release process with lots of manual steps
Chris
Am 14.04.20, 22:44 schrieb "Gabriel Bräscher" <gabrasc...@gmail.com>:
Hello Alex, hope that you are well.
Thank you very much for the reply and sorry for the (very) late reply.
I just got back to the Jenkins situation (we will release a new
CloudStack
version soon).
I totally understand that we cannot sign the packages in an automated
way,
that is far from secure.
Regarding the package building, can you please advise if it is possible
to
configure a job on builds.apache.org?
I know that there are companies using CloudStack able to donate slaves
to
help running the jobs packaging. If there is a way to contribute, please
let me know.
Additionally, I have access to create jobs, if that is ok I wold be
configuring a few jobs to help the project with CI/packaging and tests.
We
already have a lot of help with current job, however on some cases we
need
to manually test the PR, and having a way of sharing the build on the
development phase would be a big plus.
Best regards,
Gabriel.
Em qui., 12 de dez. de 2019 às 13:33, Alex Harui <aha...@adobe.com>
escreveu:
> AIUI, you can automate everything except the verification and PGP
> signing. The building and packaging doesn't have to be done on
computers
> under the RM's control, but before the RM applies his/her PGP
signature,
> the RM must convince themselves that the artifacts they are signing
are
> correct and safe. IMO the way to do that is to download the
artifacts to a
> computer under the RM's control that has the RM's PGP key on it, and
then
> run some sort of tests before signing. For the source package, it
can be
> as simple as checking out the tag into another folder and doing a
diff.
> For binaries it is much harder, but with the trend towards
reproducible
> binaries, I believe it is now practical.
>
> But then the RM has a pile of signed artifacts on some computer that
has
> to be uploaded to the distribution servers. A script can help with
that,
> though.
>
> HTH,
> -Alex
>
> On 12/12/19, 2:55 AM, "Gabriel Beims Bräscher" <gabr...@apache.org>
> wrote:
>
> Hello,
>
> I am an Apache CloudStack PMC/Committer.
>
> One of the goals that we at CloudStack have is to automatically
build
> packages and update our mirrors whenever a new release is
launched.
> I worked as release manager for CloudStack 4.12.0.0 (non-LTS) and
> assisted on 4.13.0.0 (LTS), In both cases I executed the building
> process manually (build all the deb and rpm packages and make them
> available at the ACS repository mirror). It would be great to
make it
> automated.
>
> With that in mind I am wondering If is there a way to set up
Jenkins.
> Additionally, I work at PCextreme and as a cloud provider we
would be
> happy to donate resources (virtual machine(s)), if necessary to
run
> such
> jobs.
>
> Thanks for all the help!
> Best regards,
> Gabriel.
>
> --
> Gabriel Beims Bräscher
> Apache CloudStack Committer/PMC
> The Apache Software Foundation
>
>
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2F&data=02%7C01%7Caharui%40adobe.com%7Ce72f016ca83a409a6a7808d7e10a8345%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637225307639333642&sdata=gf6rjeVocgFiOqP8tyGjJ9uuWlt93v%2F%2BCiyYM3SdXj8%3D&reserved=0
>
>
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcloudstack.apache.org%2F&data=02%7C01%7Caharui%40adobe.com%7Ce72f016ca83a409a6a7808d7e10a8345%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637225307639333642&sdata=VHfFZMF6CnMPScHhE5cii7%2Fw8vgToMamnP8Y4CWAUNc%3D&reserved=0
>
>
>
>
