GitHub Pull Request Builder - Deprecated

[Gavin McDonald]

Hi All,

For those of you that used the specific "GitHub Pull Request Builder" on
builds.a.o ,
we will not be installing it on the ci-builds.a.o.

The plugin deprecated and unmaintained [1] and it is recommended to move to
one of
the other GH plugins we have installed [2] and [3].

If you need help migrating to these other plugins, please post a new
message here
and have others here help you with it.

Thank You.

[1] - https://plugins.jenkins.io/ghprb/
[2] -
https://docs.cloudbees.com/docs/cloudbees-ci/latest/cloud-admin-guide/github-branch-source-plugin

[3] -
https://docs.cloudbees.com/docs/admin-resources/latest/plugins/pull-request-builder-for-github#pull-request-builder-for-github-usage



-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team

New Credentials for Github jobs

[Gavin McDonald]

Hi All,

For those of you waiting for the 'asf-ci' credentials - this is still not
resolved yet, and is waiting
for Cloudbees support.

However - I have created some new credentials, based off of a GH App rather
than a role account.

Look for credentials 'ASF Cloudbees Jenkins ci-builds' and give that a try
in your jobs please and see if that works for you. Let me know how it goes
for you.

-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team

Who has not migrated yet?

[Gavin McDonald]

Hi All,

Tomorrow is the deadline for migrating to ci-builds.a.o and for builds.a.o
to be turned off.

So, who has not migrated yet?
If not, why not? What is holding you up?

If you need help, ask.

If you have many jobs to migrate - please check out the script [1] which
can help you
migrate all jobs in less than 5 minutes! (I know, I've tested it!)

Are there plugins missing you need ? (except ghprb)
What else are you waiting for?

Are there outstanding tasks that Infra needs to do that might have been
missed?

Lets see if we can get off by end of day tomorrow

[1] -
https://cwiki.apache.org/confluence/display/INFRA/Migrating+Jenkins+jobs+from+Jenkins+to+Cloudbees


-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team

MAVEN - Turn your jobs off on builds.a.o

[Gavin McDonald]

Hi maven folks

You have jobs on both builds.a.o and ci-builds.a.o - have you completed
migrations? If so, please disable or delete your jobs on builds.a.o as they
are causing huge queues again. This not only affects those still using the
builds.a.o but also triggers our monitoring systems and pages our on call
folks - which seems to happen at around 3am each morning.

Thanks

-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team

Re: Who has not migrated yet?

[Enrico Olivelli]

Gavin
In Bookkeeper we haven't migrated.
I will do it today.

The switch will be easy as we are using Jenkins DSL and we already migrated
the pull request validation to Github actions

Enrico

Il Ven 14 Ago 2020, 10:33 Gavin McDonald  ha scritto:

> Hi All,
>
> Tomorrow is the deadline for migrating to ci-builds.a.o and for builds.a.o
> to be turned off.
>
> So, who has not migrated yet?
> If not, why not? What is holding you up?
>
> If you need help, ask.
>
> If you have many jobs to migrate - please check out the script [1] which
> can help you
> migrate all jobs in less than 5 minutes! (I know, I've tested it!)
>
> Are there plugins missing you need ? (except ghprb)
> What else are you waiting for?
>
> Are there outstanding tasks that Infra needs to do that might have been
> missed?
>
> Lets see if we can get off by end of day tomorrow
>
> [1] -
>
> https://cwiki.apache.org/confluence/display/INFRA/Migrating+Jenkins+jobs+from+Jenkins+to+Cloudbees
>
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>

Re: Who has not migrated yet?

[P. Ottlinger]

Hi Gavin,

can you put all XML-files of Creadur into my home directory as a ZIP?

I finished migrating Creadur RAT, but the other stuff is left to do 

It would be helpful to have these jobs as XML.

I could add them as they are needed.

As I did not work with the Token stuff I'm unable to use Uwe's script at
the moment.

Thanks,
Phil

Am 14.08.20 um 10:32 schrieb Gavin McDonald:
> Hi All,
> 
> Tomorrow is the deadline for migrating to ci-builds.a.o and for builds.a.o
> to be turned off.
> 
> So, who has not migrated yet?
> If not, why not? What is holding you up?
> 
> If you need help, ask.
> 
> If you have many jobs to migrate - please check out the script [1] which
> can help you
> migrate all jobs in less than 5 minutes! (I know, I've tested it!)
> 
> Are there plugins missing you need ? (except ghprb)
> What else are you waiting for?
> 
> Are there outstanding tasks that Infra needs to do that might have been
> missed?
> 
> Lets see if we can get off by end of day tomorrow
> 
> [1] -
> https://cwiki.apache.org/confluence/display/INFRA/Migrating+Jenkins+jobs+from+Jenkins+to+Cloudbees
> 
> 

Re: GitHub Pull Request Builder - Deprecated

[Lukasz Lenart]

pt., 14 sie 2020 o 09:31 Gavin McDonald  napisał(a):
>
> Hi All,
>
> For those of you that used the specific "GitHub Pull Request Builder" on
> builds.a.o ,
> we will not be installing it on the ci-builds.a.o.
>
> The plugin deprecated and unmaintained [1] and it is recommended to move to
> one of
> the other GH plugins we have installed [2] and [3].
>
> If you need help migrating to these other plugins, please post a new
> message here
> and have others here help you with it.
>
> Thank You.
>
> [1] - https://plugins.jenkins.io/ghprb/
> [2] -
> https://docs.cloudbees.com/docs/cloudbees-ci/latest/cloud-admin-guide/github-branch-source-plugin
>
> [3] -
> https://docs.cloudbees.com/docs/admin-resources/latest/plugins/pull-request-builder-for-github#pull-request-builder-for-github-usage

I have enabled "Build pull requests to the repository" as mentioned in
[3] and could you check if the webhook was registered for the below
repo?

https://github.com/apache/struts-site/pull/147/files


Regards
-- 
Łukasz
+ 48 606 323 122 http://www.lenart.org.pl/

Re: New Credentials for Github jobs

[Joan Touzet]

Could we get these on the ci-couchdb server for testing? Thanks.

-Joan

On 2020-08-14 3:37 a.m., Gavin McDonald wrote:

Hi All,

For those of you waiting for the 'asf-ci' credentials - this is still not
resolved yet, and is waiting
for Cloudbees support.

However - I have created some new credentials, based off of a GH App rather
than a role account.

Look for credentials 'ASF Cloudbees Jenkins ci-builds' and give that a try
in your jobs please and see if that works for you. Let me know how it goes
for you.

Migrating a project from GitLab CI, setting up GitHub Actions

[Sander Striker]

Hi,

For anyone interested in following along, or providing any guidance or
help: we're looking to migrate BuildStream (see:
https://petri.apache.org/projects)  from GitLab to GitHub, including moving
from GitLab CI to GitHub Actions (https://s.apache.org/3tp4q
).

Cheers,

Sander

Re: Who has not migrated yet?

[Andor Molnar]

Hi Gavin,

Apache ZooKeeper is pretty much done with the migration. We’re waiting for a 
working GitHub account/app to report the build status back to GitHub and we’re 
done.

Thanks,
Andor



> On 2020. Aug 14., at 10:32, Gavin McDonald  wrote:
> 
> Hi All,
> 
> Tomorrow is the deadline for migrating to ci-builds.a.o and for builds.a.o
> to be turned off.
> 
> So, who has not migrated yet?
> If not, why not? What is holding you up?
> 
> If you need help, ask.
> 
> If you have many jobs to migrate - please check out the script [1] which
> can help you
> migrate all jobs in less than 5 minutes! (I know, I've tested it!)
> 
> Are there plugins missing you need ? (except ghprb)
> What else are you waiting for?
> 
> Are there outstanding tasks that Infra needs to do that might have been
> missed?
> 
> Lets see if we can get off by end of day tomorrow
> 
> [1] -
> https://cwiki.apache.org/confluence/display/INFRA/Migrating+Jenkins+jobs+from+Jenkins+to+Cloudbees
> 
> 
> -- 
> 
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team

Re: Who has not migrated yet?

[Mark Struberg]

Hi Gavin!

I know quite a few projects who are not yet migrated.

Is it possible to take the current CI job config xmls (or exports) and send it 
to the corresponding private lists?


txs and LieGrue,
strub


> Am 14.08.2020 um 10:32 schrieb Gavin McDonald :
> 
> Hi All,
> 
> Tomorrow is the deadline for migrating to ci-builds.a.o and for builds.a.o
> to be turned off.
> 
> So, who has not migrated yet?
> If not, why not? What is holding you up?
> 
> If you need help, ask.
> 
> If you have many jobs to migrate - please check out the script [1] which
> can help you
> migrate all jobs in less than 5 minutes! (I know, I've tested it!)
> 
> Are there plugins missing you need ? (except ghprb)
> What else are you waiting for?
> 
> Are there outstanding tasks that Infra needs to do that might have been
> missed?
> 
> Lets see if we can get off by end of day tomorrow
> 
> [1] -
> https://cwiki.apache.org/confluence/display/INFRA/Migrating+Jenkins+jobs+from+Jenkins+to+Cloudbees
> 
> 
> -- 
> 
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team

Re: Who has not migrated yet?

[Martin Desruisseaux]

Apache SIS did not migrated yet, but it is fine to create a new job from 
scratch on https://ci-builds.apache.org/ (it does not have advanced or 
unusual features). How do we create a "SIS" job? I could not find a 
"Create new Job" button.


    Thanks,

        Martin

Re: Who has not migrated yet?

[Tilman Hausherr]

Am 14.08.2020 um 10:32 schrieb Gavin McDonald:

So, who has not migrated yet?



The tika project, I've just created an Infra issue.

Tilman

Re: Who has not migrated yet?

[Dave Fisher]

Hi Gavin,

I still need to migrate the Incubator JBake / custom site builds.

Would someone check and create an Incubator folder permissioned to the IPMC. We 
had a discussion on general@ a week or two ago.

I’ll look into this later today.

Regards,
Dave

Sent from my iPhone

> On Aug 14, 2020, at 1:33 AM, Gavin McDonald  wrote:
> 
> Hi All,
> 
> Tomorrow is the deadline for migrating to ci-builds.a.o and for builds.a.o
> to be turned off.
> 
> So, who has not migrated yet?
> If not, why not? What is holding you up?
> 
> If you need help, ask.
> 
> If you have many jobs to migrate - please check out the script [1] which
> can help you
> migrate all jobs in less than 5 minutes! (I know, I've tested it!)
> 
> Are there plugins missing you need ? (except ghprb)
> What else are you waiting for?
> 
> Are there outstanding tasks that Infra needs to do that might have been
> missed?
> 
> Lets see if we can get off by end of day tomorrow
> 
> [1] -
> https://cwiki.apache.org/confluence/display/INFRA/Migrating+Jenkins+jobs+from+Jenkins+to+Cloudbees
> 
> 
> -- 
> 
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team

Migrating BookKeeper to new CI

[Enrico Olivelli]

Hi,
it looks like I cannot create new jobs in https://ci-builds.apache.org/

I am listed successfully on the LDAP group (I used to manage our CI jobs on
builds.apache.org)
https://whimsy.apache.org/roster/group/hudson-jobadmin

I cannot find other information other than here
https://cwiki.apache.org/confluence/display/INFRA/Jenkins#Jenkins-HowdoIgetanaccount?

Can you also please create a "BookKeeper" folder ?

My plan is to create a "seed job" like this one
https://builds.apache.org/job/bookkeeper-seed/

I expect that this job will automatically create all of the other jobs.
Once it is on I will update the configuration of all of the generated jobs
(modifying DSL files on git)

I apologize for being so late with this

Thanks

Enrico Olivelli

Re: Migrating BookKeeper to new CI

[Enrico Olivelli]

It looks like I can work on this space
https://ci-hadoop.apache.org/

Is it the right place for BookKeeper (it is in the hadoop umbrella, like
ZooKeeper) ?

Enrico

Il giorno ven 14 ago 2020 alle ore 13:43 Enrico Olivelli <
eolive...@apache.org> ha scritto:

> Hi,
> it looks like I cannot create new jobs in https://ci-builds.apache.org/
>
> I am listed successfully on the LDAP group (I used to manage our CI jobs
> on builds.apache.org)
> https://whimsy.apache.org/roster/group/hudson-jobadmin
>
> I cannot find other information other than here
>
> https://cwiki.apache.org/confluence/display/INFRA/Jenkins#Jenkins-HowdoIgetanaccount?
>
> Can you also please create a "BookKeeper" folder ?
>
> My plan is to create a "seed job" like this one
> https://builds.apache.org/job/bookkeeper-seed/
>
> I expect that this job will automatically create all of the other jobs.
> Once it is on I will update the configuration of all of the generated jobs
> (modifying DSL files on git)
>
> I apologize for being so late with this
>
> Thanks
>
> Enrico Olivelli
>

Re: Migrating BookKeeper to new CI

[Enrico Olivelli]

It looks like I managed to create the Folder and the bookkeeper-seed jobs
https://ci-hadoop.apache.org/job/BookKeeper/job/bookkeeper-seed/

But I cannot find the "Process DSL scripts" action

Is this feature not working on new Jenkins ?

Enrico

Il giorno ven 14 ago 2020 alle ore 13:45 Enrico Olivelli <
eolive...@gmail.com> ha scritto:

> It looks like I can work on this space
> https://ci-hadoop.apache.org/
>
> Is it the right place for BookKeeper (it is in the hadoop umbrella, like
> ZooKeeper) ?
>
> Enrico
>
> Il giorno ven 14 ago 2020 alle ore 13:43 Enrico Olivelli <
> eolive...@apache.org> ha scritto:
>
>> Hi,
>> it looks like I cannot create new jobs in https://ci-builds.apache.org/
>>
>> I am listed successfully on the LDAP group (I used to manage our CI jobs
>> on builds.apache.org)
>> https://whimsy.apache.org/roster/group/hudson-jobadmin
>>
>> I cannot find other information other than here
>>
>> https://cwiki.apache.org/confluence/display/INFRA/Jenkins#Jenkins-HowdoIgetanaccount?
>>
>> Can you also please create a "BookKeeper" folder ?
>>
>> My plan is to create a "seed job" like this one
>> https://builds.apache.org/job/bookkeeper-seed/
>>
>> I expect that this job will automatically create all of the other jobs.
>> Once it is on I will update the configuration of all of the generated
>> jobs (modifying DSL files on git)
>>
>> I apologize for being so late with this
>>
>> Thanks
>>
>> Enrico Olivelli
>>
>

Re: New Credentials for Github jobs

[Andor Molnar]

Hi Gavin,

Looks like that the new credentials don’t have rights to update build status 
either:

https://ci-hadoop.apache.org/view/ZooKeeper/job/ZooKeeper%20PreCommit%20GitHub%20PR/view/change-requests/job/PR-1429/2/console

—
Started by user Andor Molnar
09:38:53 Connecting to https://api.github.com using 76885/** (ASF Cloudbees 
Jenkins ci-hadoop)

…

Could not update commit status, please check if your scan credentials belong to 
a member of the organization or a collaborator of the repository and 
repo:status scope is selected


GitHub has been notified of this commit’s build result

ERROR: script returned exit code 1
Finished: FAILURE
—

Andor



> On 2020. Aug 14., at 9:37, Gavin McDonald  wrote:
> 
> Hi All,
> 
> For those of you waiting for the 'asf-ci' credentials - this is still not
> resolved yet, and is waiting
> for Cloudbees support.
> 
> However - I have created some new credentials, based off of a GH App rather
> than a role account.
> 
> Look for credentials 'ASF Cloudbees Jenkins ci-builds' and give that a try
> in your jobs please and see if that works for you. Let me know how it goes
> for you.
> 
> -- 
> 
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team

Re: New Credentials for Github jobs

[Robert Munteanu]

On Fri, 2020-08-14 at 14:36 +0200, Andor Molnar wrote:
> Looks like that the new credentials don’t have rights to update build
> status either

FWIW for the Sling builds I just created a personal access token with
'repo' scope. Kind of surprising to see a personal avatar commenting
left and right, but that works with very little hassle.

Thanks,
Robert

Re: New Credentials for Github jobs

[Andor Molnar]

Thanks Robert, that works.
Nice to see my face next to ZooKeeper CI builds. :)


> On 2020. Aug 14., at 15:13, Robert Munteanu  wrote:
> 
> On Fri, 2020-08-14 at 14:36 +0200, Andor Molnar wrote:
>> Looks like that the new credentials don’t have rights to update build
>> status either
> 
> FWIW for the Sling builds I just created a personal access token with
> 'repo' scope. Kind of surprising to see a personal avatar commenting
> left and right, but that works with very little hassle.
> 
> Thanks,
> Robert
> 

Re: Who has not migrated yet?

[Dave Brondsema]

Can you create a folder for Allura jobs?

On 8/14/20 4:32 AM, Gavin McDonald wrote:
> Hi All,
> 
> Tomorrow is the deadline for migrating to ci-builds.a.o and for builds.a.o
> to be turned off.
> 
> So, who has not migrated yet?
> If not, why not? What is holding you up?
> 
> If you need help, ask.
> 
> If you have many jobs to migrate - please check out the script [1] which
> can help you
> migrate all jobs in less than 5 minutes! (I know, I've tested it!)
> 
> Are there plugins missing you need ? (except ghprb)
> What else are you waiting for?
> 
> Are there outstanding tasks that Infra needs to do that might have been
> missed?
> 
> Lets see if we can get off by end of day tomorrow
> 
> [1] -
> https://cwiki.apache.org/confluence/display/INFRA/Migrating+Jenkins+jobs+from+Jenkins+to+Cloudbees
> 
> 


-- 
Dave Brondsema : d...@brondsema.net
http://www.brondsema.net : personal
http://www.splike.com : programming
  <><

Re: New Credentials for Github jobs

[Gavin McDonald]

Hi Andor

On Fri, Aug 14, 2020 at 2:36 PM Andor Molnar  wrote:

> Hi Gavin,
>
> Looks like that the new credentials don’t have rights to update build
> status either:
>

I fixed that now, try again please.



>
>
> https://ci-hadoop.apache.org/view/ZooKeeper/job/ZooKeeper%20PreCommit%20GitHub%20PR/view/change-requests/job/PR-1429/2/console
>
> —
> Started by user Andor Molnar
> 09:38:53 Connecting to https://api.github.com using 76885/** (ASF
> Cloudbees Jenkins ci-hadoop)
>
> …
>
> Could not update commit status, please check if your scan credentials
> belong to a member of the organization or a collaborator of the repository
> and repo:status scope is selected
>
>
> GitHub has been notified of this commit’s build result
>
> ERROR: script returned exit code 1
> Finished: FAILURE
> —
>
> Andor
>
>
>
> > On 2020. Aug 14., at 9:37, Gavin McDonald  wrote:
> >
> > Hi All,
> >
> > For those of you waiting for the 'asf-ci' credentials - this is still not
> > resolved yet, and is waiting
> > for Cloudbees support.
> >
> > However - I have created some new credentials, based off of a GH App
> rather
> > than a role account.
> >
> > Look for credentials 'ASF Cloudbees Jenkins ci-builds' and give that a
> try
> > in your jobs please and see if that works for you. Let me know how it
> goes
> > for you.
> >
> > --
> >
> > *Gavin McDonald*
> > Systems Administrator
> > ASF Infrastructure Team
>
>

-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team

Re: Migrating a project from GitLab CI, setting up GitHub Actions

[Jarek Potiuk]

Hey Sander,

Happy to help.

Side comment: I love the way GitHub actions evolve and I have quite an
experience with running and optimizing our builds for Apache Airflow. We
are really happy with it especially comparing to Travis CI  we used
previously. We've been waiting > 1 year (!!!) for GitLab to add running CI
jobs for fork pull requests (
https://gitlab.com/gitlab-org/gitlab/-/issues/5667 - the issue is already 2
years old and keeps on being pushed to the next release with every release
so we stopped waiting for it.

The speed with which Github Actions adds new features is impressive, to say
the least, and their public roadmap
https://github.com/github/roadmap/projects/1 is even more impressive).

I am happy to provide help with my experience (I will subscribe to your
devlist now).

I am right now in the process of testing a big overhaul of the way how
Apache Airflow uses Github Actions.  Mostly it is about consistency and
optimizations of waiting time + decreasing the pressure on the Github
Actions workers we use (we have rather long and comprehensive tests and big
matrix of jobs). You can see some (almost completed) work in progress here
https://github.com/potiuk/airflow/tree/master/.github/workflows.

Along the way I am also finishing up a Github Action that is going to make
full use of recently released "workflow_run" type of triggers - that
enabled automated canceling of duplicated Pull Request builds from forks,
which was a major pain for us (and so far we only workaround it with
cancel run scheduled every 5 minutes - which was pretty nasty :(. With the
new "workflow_run" it seems possible, though a bit complex, that's why I
took an existing cancel-workflow action written by someone else and added
full support for workflow_runs (workflow_runs have been added ~ 1 week ago
or so so they are rather fresh). The action is here:
https://github.com/potiuk/cancel-workflow-runs#cancel-duplicate-jobs-for-triggered-workflow
(master
version is what I am testing right now and will release v2 shortly with
full support for workflow_run but the documentation is already rather
comprehensive and contains examples and explanation how things work

I am also helping the Apache Beam team in their Github Action migration, so
happy to help you as well :). I am also working on handling flaky tests
(and sharing experiences with BEam team as well). We've introduced concept
of "Quarantined tests" and I am looking into using
https://github.com/GoogleCloudPlatform/flaky-service - a service developed
by Google Cloud Platform to manage flaky tests (It has Gihub Action
integration out-of-the box).

You might also find useful the architecture of our integration between
Github Actions and DockerHub and the approach we've taken so far (I
contributed it to infra some time ago
https://cwiki.apache.org/confluence/display/INFRA/Github+Actions+to+DockerHub).
It will be slightly updated with the Overhaul I am working on, so I am
going to update it shortly, the in-progress version of it is in
https://github.com/potiuk/airflow/blob/master/CI.rst

J.



On Fri, Aug 14, 2020 at 12:00 PM Sander Striker 
wrote:

> Hi,
>
> For anyone interested in following along, or providing any guidance or
> help: we're looking to migrate BuildStream (see:
> https://petri.apache.org/projects)  from GitLab to GitHub, including
> moving
> from GitLab CI to GitHub Actions (https://s.apache.org/3tp4q
> ).
>
> Cheers,
>
> Sander
>


-- 

Jarek Potiuk
Polidea  | Principal Software Engineer

M: +48 660 796 129 <+48660796129>
[image: Polidea] 

Re: New Credentials for Github jobs

[Gavin McDonald]

Hi Joan,

On Fri, Aug 14, 2020 at 11:30 AM Joan Touzet  wrote:

> Could we get these on the ci-couchdb server for testing? Thanks.
>

Sure thing, added a new GH App and credentials for it are on ci-couchdb
Master

HTH


>
> -Joan
>
> On 2020-08-14 3:37 a.m., Gavin McDonald wrote:
> > Hi All,
> >
> > For those of you waiting for the 'asf-ci' credentials - this is still not
> > resolved yet, and is waiting
> > for Cloudbees support.
> >
> > However - I have created some new credentials, based off of a GH App
> rather
> > than a role account.
> >
> > Look for credentials 'ASF Cloudbees Jenkins ci-builds' and give that a
> try
> > in your jobs please and see if that works for you. Let me know how it
> goes
> > for you.
> >
>


-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team

Re: Who has not migrated yet?

[Gavin McDonald]

Hi Dave,

On Fri, Aug 14, 2020 at 4:56 PM Dave Brondsema  wrote:

> Can you create a folder for Allura jobs?
>
>
Sure, done!



> On 8/14/20 4:32 AM, Gavin McDonald wrote:
> > Hi All,
> >
> > Tomorrow is the deadline for migrating to ci-builds.a.o and for
> builds.a.o
> > to be turned off.
> >
> > So, who has not migrated yet?
> > If not, why not? What is holding you up?
> >
> > If you need help, ask.
> >
> > If you have many jobs to migrate - please check out the script [1] which
> > can help you
> > migrate all jobs in less than 5 minutes! (I know, I've tested it!)
> >
> > Are there plugins missing you need ? (except ghprb)
> > What else are you waiting for?
> >
> > Are there outstanding tasks that Infra needs to do that might have been
> > missed?
> >
> > Lets see if we can get off by end of day tomorrow
> >
> > [1] -
> >
> https://cwiki.apache.org/confluence/display/INFRA/Migrating+Jenkins+jobs+from+Jenkins+to+Cloudbees
> >
> >
>
>
> --
> Dave Brondsema : d...@brondsema.net
> http://www.brondsema.net : personal
> http://www.splike.com : programming
>   <><
>


-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team

Re: GitHub Pull Request Builder - Deprecated

[Ismael Juma]

Hi Gavin,

Do these plugins (`2` and `3`) allow multiple pull request builders to be
configured? I couldn't find it in the documentation. This would be a
significant regression for Apache Kafka, unfortunately. Is there a
possibility that we install `1` with a plan to remove in 3-6 months? That
would give projects a bit of time to figure out how to migrate without
causing major disruption to the existing flow.

Ismael

On Fri, Aug 14, 2020 at 12:31 AM Gavin McDonald 
wrote:

> Hi All,
>
> For those of you that used the specific "GitHub Pull Request Builder" on
> builds.a.o ,
> we will not be installing it on the ci-builds.a.o.
>
> The plugin deprecated and unmaintained [1] and it is recommended to move to
> one of
> the other GH plugins we have installed [2] and [3].
>
> If you need help migrating to these other plugins, please post a new
> message here
> and have others here help you with it.
>
> Thank You.
>
> [1] - https://plugins.jenkins.io/ghprb/
> [2] -
>
> https://docs.cloudbees.com/docs/cloudbees-ci/latest/cloud-admin-guide/github-branch-source-plugin
>
> [3] -
>
> https://docs.cloudbees.com/docs/admin-resources/latest/plugins/pull-request-builder-for-github#pull-request-builder-for-github-usage
>
>
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>

Re: Who has not migrated yet?

[Gavin McDonald]

Hi Martin,

On Fri, Aug 14, 2020 at 12:26 PM Martin Desruisseaux <
martin.desruisse...@geomatys.com> wrote:

> Apache SIS did not migrated yet, but it is fine to create a new job from
> scratch on https://ci-builds.apache.org/ (it does not have advanced or
> unusual features). How do we create a "SIS" job? I could not find a
> "Create new Job" button.
>

Right, the folder for the project needed to be created first.

Look for the SIS folder now where you can create jobs

HTH


>
>  Thanks,
>
>  Martin
>
>
>

-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team

Re: Who has not migrated yet?

[Gavin McDonald]

Hi Dave,


On Fri, Aug 14, 2020 at 12:42 PM Dave Fisher  wrote:

> Hi Gavin,
>
> I still need to migrate the Incubator JBake / custom site builds.
>
> Would someone check and create an Incubator folder permissioned to the
> IPMC. We had a discussion on general@ a week or two ago.
>

Folder created, IPMC have access


>
> I’ll look into this later today.
>
> Regards,
> Dave
>
> Sent from my iPhone
>
> > On Aug 14, 2020, at 1:33 AM, Gavin McDonald 
> wrote:
> >
> > Hi All,
> >
> > Tomorrow is the deadline for migrating to ci-builds.a.o and for
> builds.a.o
> > to be turned off.
> >
> > So, who has not migrated yet?
> > If not, why not? What is holding you up?
> >
> > If you need help, ask.
> >
> > If you have many jobs to migrate - please check out the script [1] which
> > can help you
> > migrate all jobs in less than 5 minutes! (I know, I've tested it!)
> >
> > Are there plugins missing you need ? (except ghprb)
> > What else are you waiting for?
> >
> > Are there outstanding tasks that Infra needs to do that might have been
> > missed?
> >
> > Lets see if we can get off by end of day tomorrow
> >
> > [1] -
> >
> https://cwiki.apache.org/confluence/display/INFRA/Migrating+Jenkins+jobs+from+Jenkins+to+Cloudbees
> >
> >
> > --
> >
> > *Gavin McDonald*
> > Systems Administrator
> > ASF Infrastructure Team
>
>

-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team

Re: Who has not migrated yet?

[Kevin Risden]

Can a Knox folder be created?

Thanks!

Kevin Risden



On Fri, Aug 14, 2020 at 1:49 PM Gavin McDonald  wrote:

> Hi Dave,
>
>
> On Fri, Aug 14, 2020 at 12:42 PM Dave Fisher  wrote:
>
> > Hi Gavin,
> >
> > I still need to migrate the Incubator JBake / custom site builds.
> >
> > Would someone check and create an Incubator folder permissioned to the
> > IPMC. We had a discussion on general@ a week or two ago.
> >
>
> Folder created, IPMC have access
>
>
> >
> > I’ll look into this later today.
> >
> > Regards,
> > Dave
> >
> > Sent from my iPhone
> >
> > > On Aug 14, 2020, at 1:33 AM, Gavin McDonald 
> > wrote:
> > >
> > > Hi All,
> > >
> > > Tomorrow is the deadline for migrating to ci-builds.a.o and for
> > builds.a.o
> > > to be turned off.
> > >
> > > So, who has not migrated yet?
> > > If not, why not? What is holding you up?
> > >
> > > If you need help, ask.
> > >
> > > If you have many jobs to migrate - please check out the script [1]
> which
> > > can help you
> > > migrate all jobs in less than 5 minutes! (I know, I've tested it!)
> > >
> > > Are there plugins missing you need ? (except ghprb)
> > > What else are you waiting for?
> > >
> > > Are there outstanding tasks that Infra needs to do that might have been
> > > missed?
> > >
> > > Lets see if we can get off by end of day tomorrow
> > >
> > > [1] -
> > >
> >
> https://cwiki.apache.org/confluence/display/INFRA/Migrating+Jenkins+jobs+from+Jenkins+to+Cloudbees
> > >
> > >
> > > --
> > >
> > > *Gavin McDonald*
> > > Systems Administrator
> > > ASF Infrastructure Team
> >
> >
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>

Re: Who has not migrated yet?

[Dave Fisher]

Hi Gavin,

Thank you very much.

I’m using Uwe’s script from  
https://cwiki.apache.org/confluence/display/INFRA/Migrating+jobs+from+Jenkins+to+Cloudbees

I’m looking for the API Tokens.

I asked on Slack as well.

Regards,
Dave

> On Aug 14, 2020, at 10:48 AM, Gavin McDonald  wrote:
> 
> Hi Dave,
> 
> 
> On Fri, Aug 14, 2020 at 12:42 PM Dave Fisher  wrote:
> 
>> Hi Gavin,
>> 
>> I still need to migrate the Incubator JBake / custom site builds.
>> 
>> Would someone check and create an Incubator folder permissioned to the
>> IPMC. We had a discussion on general@ a week or two ago.
>> 
> 
> Folder created, IPMC have access
> 
> 
>> 
>> I’ll look into this later today.
>> 
>> Regards,
>> Dave
>> 
>> Sent from my iPhone
>> 
>>> On Aug 14, 2020, at 1:33 AM, Gavin McDonald 
>> wrote:
>>> 
>>> Hi All,
>>> 
>>> Tomorrow is the deadline for migrating to ci-builds.a.o and for
>> builds.a.o
>>> to be turned off.
>>> 
>>> So, who has not migrated yet?
>>> If not, why not? What is holding you up?
>>> 
>>> If you need help, ask.
>>> 
>>> If you have many jobs to migrate - please check out the script [1] which
>>> can help you
>>> migrate all jobs in less than 5 minutes! (I know, I've tested it!)
>>> 
>>> Are there plugins missing you need ? (except ghprb)
>>> What else are you waiting for?
>>> 
>>> Are there outstanding tasks that Infra needs to do that might have been
>>> missed?
>>> 
>>> Lets see if we can get off by end of day tomorrow
>>> 
>>> [1] -
>>> 
>> https://cwiki.apache.org/confluence/display/INFRA/Migrating+Jenkins+jobs+from+Jenkins+to+Cloudbees
>>> 
>>> 
>>> --
>>> 
>>> *Gavin McDonald*
>>> Systems Administrator
>>> ASF Infrastructure Team
>> 
>> 
> 
> -- 
> 
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team

Process Jenkins DSL action

[Enrico Olivelli]

Hi,
I am migrating Bookkeeper jobs and it looks like there is no 'Process DSL
actin'

We have a seed job that creates all of the other jobs

I have already created the new seed but actually it is useless.

Any idea?

Enrico

Re: Process Jenkins DSL action

[Gavin McDonald]

Hi Enrico!

On Fri, Aug 14, 2020 at 8:41 PM Enrico Olivelli  wrote:

> Hi,
> I am migrating Bookkeeper jobs and it looks like there is no 'Process DSL
> actin'
>
> We have a seed job that creates all of the other jobs
>
> I have already created the new seed but actually it is useless.
>
> Any idea?
>

Right, so a part of this migration process involves finding out what
plugins might be
needed for Jobs, and installing them as we come across them.

I just installed the Job DSL plugin and your Process DSL option should now
be available for you.

Let me know how you get on


> Enrico
>


-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team

Re: Who has not migrated yet?

[Martin Desruisseaux]

Le 14/08/2020 à 19:45, Gavin McDonald a écrit :


Right, the folder for the project needed to be created first.
Look for the SIS folder now where you can create jobs 


Thanks! Job created.

    Martin

Re: Who has not migrated yet?

[P. Ottlinger]

Hi Gav,

hopefully all of Creadur's projects are migrated . puh -
I created a new infra ticket
(https://issues.apache.org/jira/browse/INFRA-20688) to get some of the
older projects moved over to Gitbox, but all builds seem to work:

https://ci-builds.apache.org/job/Creadur/

Thanks for the reminder!

Phil

Re: [ci-builds] GitHub credentials

[Richard Eckart de Castilho]

Hi,

> On 30. Jul 2020, at 05:42, Chris Lambertus  wrote:
> 
> There is no policy per-se, we have done this in the past, specifically for 
> repo:status tokens;
> 
> Please create an infra jira ticket. Unfortunately, it’s an extremely manual 
> process on our end, and requires Infra to create and maintain an account and 
> password+token for each project, so it’s something we try to avoid unless 
> absolutely necessary. 

In which case would such a thing ever be "absolutely necessary"? 

However, I have come to find it very very convenient to be able to visit a PR 
and see how it fares, in particular since I like setting up repos to not permit 
merges unless the CI gives its thumbs up - which is of course also not 
absolutely necessary - but useful.

Is there any chance there might be a more generic solution than using personal 
tokens or high-effort investment of INFRA to the problem of setting the commit 
status on GitHub from ci-builds in the near future?

Cheers,

-- Richard

Re: [ci-builds] GitHub credentials

[Chris Lambertus]

> On Aug 14, 2020, at 4:01 PM, Richard Eckart de Castilho  
> wrote:
> 
> Hi,
> 
>> On 30. Jul 2020, at 05:42, Chris Lambertus  wrote:
>> 
>> There is no policy per-se, we have done this in the past, specifically for 
>> repo:status tokens;
>> 
>> Please create an infra jira ticket. Unfortunately, it’s an extremely manual 
>> process on our end, and requires Infra to create and maintain an account and 
>> password+token for each project, so it’s something we try to avoid unless 
>> absolutely necessary. 
> 
> In which case would such a thing ever be "absolutely necessary"? 

If your project can justify the extreme need of an additional token because 
your project has so many build requests that exceed the rate limits of the 
asf-ci token Infra already provides, we'll evaluate it. We would prefer you 
discuss the needs of your build with Infra so we can assess and allocate 
resources as appropriate.


> 
> However, I have come to find it very very convenient to be able to visit a PR 
> and see how it fares, in particular since I like setting up repos to not 
> permit merges unless the CI gives its thumbs up - which is of course also not 
> absolutely necessary - but useful.
> 
> Is there any chance there might be a more generic solution than using 
> personal tokens or high-effort investment of INFRA to the problem of setting 
> the commit status on GitHub from ci-builds in the near future?
> 
> Cheers,
> 
> -- Richard

Re: Process Jenkins DSL action

[Enrico Olivelli]

Gavin,
thank you.

Now I have this second problem, it looks like that in the new Jenkins every
Jenkins DSL script must be approved.
see
https://ci-hadoop.apache.org/job/BookKeeper/job/bookkeeper-seed/3/console

I am not sure how can we make it work, the help reads "New or modified
scripts must either be approved by an Jenkins administrator before they can
be used or they must be run in the restricted sandbox."

> git config core.sparsecheckout # timeout=10
 > git checkout -f 7645cb839761b9dde6462c28a87a0d1c428bf5fa # timeout=10
Commit message: "Bookie Client add quarantine ratio when error count
exceed threshold"
 > git rev-list --no-walk 7645cb839761b9dde6462c28a87a0d1c428bf5fa # timeout=10
Processing DSL script .test-infra/jenkins/job_bookkeeper_codecoverage.groovy
ERROR: script not yet approved for use
Finished: FAILURE


If I enable the "Groovy sandbox", then I see this second error

Commit message: "Bookie Client add quarantine ratio when error count
exceed threshold"
 > git rev-list --no-walk 7645cb839761b9dde6462c28a87a0d1c428bf5fa # timeout=10

ERROR: You must configure the DSL job to run as a specific user in
order to use the Groovy sandbox.
Finished: FAILURE


Thank you

Enrico

Il giorno ven 14 ago 2020 alle ore 20:57 Gavin McDonald <
gmcdon...@apache.org> ha scritto:

> Hi Enrico!
>
> On Fri, Aug 14, 2020 at 8:41 PM Enrico Olivelli 
> wrote:
>
> > Hi,
> > I am migrating Bookkeeper jobs and it looks like there is no 'Process DSL
> > actin'
> >
> > We have a seed job that creates all of the other jobs
> >
> > I have already created the new seed but actually it is useless.
> >
> > Any idea?
> >
>
> Right, so a part of this migration process involves finding out what
> plugins might be
> needed for Jobs, and installing them as we come across them.
>
> I just installed the Job DSL plugin and your Process DSL option should now
> be available for you.
>
> Let me know how you get on
>
>
> > Enrico
> >
>
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>