Re: Jenkins

[Gabriel Bräscher]

Hello Alex, hope that you are well.

Thank you very much for the reply and sorry for the (very) late reply.

I just got back to the Jenkins situation (we will release a new CloudStack
version soon).
I totally understand that we cannot sign the packages in an automated way,
that is far from secure.

Regarding the package building, can you please advise if it is possible to
configure a job on builds.apache.org?
I know that there are companies using CloudStack able to donate slaves to
help running the jobs packaging. If there is a way to contribute, please
let me know.

Additionally, I have access to create jobs, if that is ok I wold be
configuring a few jobs to help the project with CI/packaging and tests. We
already have a lot of help with current job, however on some cases we need
to manually test the PR, and having a way of sharing the build on the
development phase would be a big plus.

Best regards,
Gabriel.

Em qui., 12 de dez. de 2019 às 13:33, Alex Harui 
escreveu:

> AIUI, you can automate everything except the verification and PGP
> signing.  The building and packaging doesn't have to be done on computers
> under the RM's control, but before the RM applies his/her PGP signature,
> the RM must convince themselves that the artifacts they are signing are
> correct and safe.  IMO the way to do that is to download the artifacts to a
> computer under the RM's control that has the RM's PGP key on it, and then
> run some sort of tests before signing.  For the source package, it can be
> as simple as checking out the tag into another folder and doing a diff.
> For binaries it is much harder, but with the trend towards reproducible
> binaries, I believe it is now practical.
>
> But then the RM has a pile of signed artifacts on some computer that has
> to be uploaded to the distribution servers.  A script can help with that,
> though.
>
> HTH,
> -Alex
>
> On 12/12/19, 2:55 AM, "Gabriel Beims Bräscher" 
> wrote:
>
> Hello,
>
> I am an Apache CloudStack PMC/Committer.
>
> One of the goals that we at CloudStack have is to automatically build
> packages and update our mirrors whenever a new release is launched.
> I worked as release manager for CloudStack 4.12.0.0 (non-LTS) and
> assisted on 4.13.0.0 (LTS), In both cases I executed the building
> process manually (build all the deb and rpm packages and make them
> available at the ACS repository mirror). It would be great to make it
> automated.
>
> With that in mind I am wondering If is there a way to set up Jenkins.
> Additionally, I work at PCextreme and as a cloud provider we would be
> happy to donate resources (virtual machine(s)), if necessary to run
> such
> jobs.
>
> Thanks for all the help!
> Best regards,
> Gabriel.
>
> --
> Gabriel Beims Bräscher
> Apache CloudStack Committer/PMC
> The Apache Software Foundation
>
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2F&data=02%7C01%7Caharui%40adobe.com%7C2776a1f593ae42724ff208d77ef1bc38%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C1%7C637117449021619621&sdata=rJarxm3rQh2Kh9oirMkQWWISQ4WquxJrO6Jr%2F2chWak%3D&reserved=0
>
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcloudstack.apache.org%2F&data=02%7C01%7Caharui%40adobe.com%7C2776a1f593ae42724ff208d77ef1bc38%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C1%7C637117449021619621&sdata=jCkeUb804f7jntV9hCgVCa7cIS6MjKuN%2B4NLqgOyzIc%3D&reserved=0
>
>
>
>

Re: Jenkins

[Alex Harui]

Hello Gabriel,

I’m just a user of the service and not part of the group that runs it.  You can 
read more about this service here: 
https://cwiki.apache.org/confluence/display/INFRA/Jenkins

I’m not knowledgeable enough about the service to say what you can’t do.  I was 
only saying explaining how my project builds releases on shared machines, which 
we do on machines outside of builds.a.o.  Because the machines in build.a.o are 
shared with other projects there may be technical or social limits on how many 
resources your jobs can use.  We don’t have that problem on our own machines, 
but then we have to spend time keeping them running.

Good luck,
-Alex

From: Gabriel Bräscher 
Date: Tuesday, April 14, 2020 at 1:44 PM
To: Alex Harui 
Cc: "builds@apache.org" , "gabr...@apache.org" 

Subject: Re: Jenkins

Hello Alex, hope that you are well.

Thank you very much for the reply and sorry for the (very) late reply.

I just got back to the Jenkins situation (we will release a new CloudStack 
version soon).
I totally understand that we cannot sign the packages in an automated way, that 
is far from secure.

Regarding the package building, can you please advise if it is possible to 
configure a job on 
builds.apache.org?
I know that there are companies using CloudStack able to donate slaves to help 
running the jobs packaging. If there is a way to contribute, please let me know.

Additionally, I have access to create jobs, if that is ok I wold be configuring 
a few jobs to help the project with CI/packaging and tests. We already have a 
lot of help with current job, however on some cases we need to manually test 
the PR, and having a way of sharing the build on the development phase would be 
a big plus.
Best regards,
Gabriel.

Em qui., 12 de dez. de 2019 às 13:33, Alex Harui 
mailto:aha...@adobe.com>> escreveu:
AIUI, you can automate everything except the verification and PGP signing.  The 
building and packaging doesn't have to be done on computers under the RM's 
control, but before the RM applies his/her PGP signature, the RM must convince 
themselves that the artifacts they are signing are correct and safe.  IMO the 
way to do that is to download the artifacts to a computer under the RM's 
control that has the RM's PGP key on it, and then run some sort of tests before 
signing.  For the source package, it can be as simple as checking out the tag 
into another folder and doing a diff.  For binaries it is much harder, but with 
the trend towards reproducible binaries, I believe it is now practical.

But then the RM has a pile of signed artifacts on some computer that has to be 
uploaded to the distribution servers.  A script can help with that, though.

HTH,
-Alex

On 12/12/19, 2:55 AM, "Gabriel Beims Bräscher" 
mailto:gabr...@apache.org>> wrote:

Hello,

I am an Apache CloudStack PMC/Committer.

One of the goals that we at CloudStack have is to automatically build
packages and update our mirrors whenever a new release is launched.
I worked as release manager for CloudStack 4.12.0.0 (non-LTS) and
assisted on 4.13.0.0 (LTS), In both cases I executed the building
process manually (build all the deb and rpm packages and make them
available at the ACS repository mirror). It would be great to make it
automated.

With that in mind I am wondering If is there a way to set up Jenkins.
Additionally, I work at PCextreme and as a cloud provider we would be
happy to donate resources (virtual machine(s)), if necessary to run such
jobs.

Thanks for all the help!
Best regards,
Gabriel.

--
Gabriel Beims Bräscher
Apache CloudStack Committer/PMC
The Apache Software Foundation

https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2F&data=02%7C01%7Caharui%40adobe.com%7C2776a1f593ae42724ff208d77ef1bc38%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C1%7C637117449021619621&sdata=rJarxm3rQh2Kh9oirMkQWWISQ4WquxJrO6Jr%2F2chWak%3D&reserved=0

https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcloudstack.apache.org%2F&data=02%7C01%7Caharui%40adobe.com%7C2776a1f593ae42724ff208d77ef1bc38%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C1%7C637117449021619621&sdata=jCkeUb804f7jntV9hCgVCa7cIS6MjKuN%2B4NLqgOyzIc%3D&reserved=0

Re: How can we customise jenkins email content?

[Matt Sicker]

Have you tried adding a config file through
https://builds.apache.org/job/JOB_NAME/configfiles/ ? I see there are
options for the extended email publisher groovy and jelly templates
there.

On Sat, 11 Apr 2020 at 15:02, Mick Semb Wever  wrote:
>
> > ${FILE,path="email.txt"}
>
>
> Ah gotcha, missed that. Yeah that will work :-) Thanks for saying it again.



-- 
Matt Sicker 

Re: How can we customise jenkins email content?

[Mick Semb Wever]

> Have you tried adding a config file through
> https://builds.apache.org/job/JOB_NAME/configfiles/ ?


Hi Matt, that must be from a plugin as I can't find any such url under
the jobs in ci-cassandra.apache.org


Otherwise there is value in having it scripted, though i did not wish
to delve into xsl ever again.

With the help I got here I've managed to mangle together the following
text report format:
https://lists.apache.org/thread.html/r80d13f7af706bf8dfbf2387fab46004c1fbd3917b7bc339c49e69aa8%40%3Cbuilds.cassandra.apache.org%3E

Re: Jenkins

[Christofer Dutz]

Hi Gabriel,

I just have to jump in here ... as I saw you were only getting feedback from 
one side.

I did see that Cloudstack generally seems to be built with Maven, so I can give 
some input

There is absolutely nothing wrong with building dev snapshots on a CI server 
... that's one of the things they were built for.

However when talking about releases the situation is different. Because you 
need a real persons credentials for:

- Pushing to Git repos
- Signing artifacts
- Deploying artifact to any form of remote repos

So these are the parts you can't automate and a RM will have to jump in 
manually.

So what's typically a 3 command release process on an ordinary maven project:

- branch
- prepare
- release

Has to be split up and divided into a sequence of steps where some can be done 
"automatic" (by clicking a button) and some have to be done manually.

I would also strongly suggest not to use the Apache Royale release process as a 
reference. It was intended to simplify things and currently is a process of 
about 100 steps that have to be executed in sequence ... some by clicking a 
button ... some by logging in on the CI server (which would probably be a 
problem on shared ASF build nodes).

I doubt this is something you would like to do. I. strongly suggest not to.

So please keep in mind that you can't have any non-technical user credentials 
on a shared CI server node ... you could probably do that if you have a 
dedicated Release Manager machine (Sort of you start it up, provide the 
credentials needed and after the Release the VM is discarded) but it's totally 
out of the question for a shared machine. And as Apache doesn't have technical 
users outside of infra, well I guess the only options you have are:

- dedicated RM machine
- very lengthy release process with lots of manual steps

Chris






Am 14.04.20, 22:44 schrieb "Gabriel Bräscher" :

Hello Alex, hope that you are well.

Thank you very much for the reply and sorry for the (very) late reply.

I just got back to the Jenkins situation (we will release a new CloudStack
version soon).
I totally understand that we cannot sign the packages in an automated way,
that is far from secure.

Regarding the package building, can you please advise if it is possible to
configure a job on builds.apache.org?
I know that there are companies using CloudStack able to donate slaves to
help running the jobs packaging. If there is a way to contribute, please
let me know.

Additionally, I have access to create jobs, if that is ok I wold be
configuring a few jobs to help the project with CI/packaging and tests. We
already have a lot of help with current job, however on some cases we need
to manually test the PR, and having a way of sharing the build on the
development phase would be a big plus.

Best regards,
Gabriel.

Em qui., 12 de dez. de 2019 às 13:33, Alex Harui 
escreveu:

> AIUI, you can automate everything except the verification and PGP
> signing.  The building and packaging doesn't have to be done on computers
> under the RM's control, but before the RM applies his/her PGP signature,
> the RM must convince themselves that the artifacts they are signing are
> correct and safe.  IMO the way to do that is to download the artifacts to 
a
> computer under the RM's control that has the RM's PGP key on it, and then
> run some sort of tests before signing.  For the source package, it can be
> as simple as checking out the tag into another folder and doing a diff.
> For binaries it is much harder, but with the trend towards reproducible
> binaries, I believe it is now practical.
>
> But then the RM has a pile of signed artifacts on some computer that has
> to be uploaded to the distribution servers.  A script can help with that,
> though.
>
> HTH,
> -Alex
>
> On 12/12/19, 2:55 AM, "Gabriel Beims Bräscher" 
> wrote:
>
> Hello,
>
> I am an Apache CloudStack PMC/Committer.
>
> One of the goals that we at CloudStack have is to automatically build
> packages and update our mirrors whenever a new release is launched.
> I worked as release manager for CloudStack 4.12.0.0 (non-LTS) and
> assisted on 4.13.0.0 (LTS), In both cases I executed the building
> process manually (build all the deb and rpm packages and make them
> available at the ACS repository mirror). It would be great to make it
> automated.
>
> With that in mind I am wondering If is there a way to set up Jenkins.
> Additionally, I work at PCextreme and as a cloud provider we would be
> happy to donate resources (virtual machine(s)), if necessary to run
> such
> jobs.
>
> Thanks for all the help!
> Best regards,
> Gabriel.
>
> --
> Gabriel Beims