Re: broken builds taking up resources

The Cassandra dtest builds take ~12 hours. The unit tests over an hour We are looking into parallelising these, but work hasn't started on that yet. We recently parallelised a number of the unit test builds, and added pipeline builds, and subsequently builds have been crashing with full disks. Ye

Re: broken builds taking up resources

> Is there some way we can improve the visibility into disk usage on the > build nodes? How full they are? And what projects are taking up space? > Does jenkins provide this info? Or could infra dump a `du …` report > somewhere? There are two Jenkins plugins that help with this situation

How can we customise jenkins email content?

Have any other projects customised the format of the email body that jenkins sends? Using the emailext plugin, the email content can be defined by a jelly script, and there's a few from that plugin to pick from. See for example https://github.com/apache/cassandra/blob/trunk/.jenkins/Jenkinsfile#L

Re: How can we customise jenkins email content?

Thanks Matt and Alex, > To elaborate further, if you look through the global variables available, > you should be able to obtain all the info you’d use in a Jelly template > without the maintenance hassle. A bit of background to the problem… The Cassandra community wants a permanent record of

Re: How can we customise jenkins email content?

> ${FILE,path="email.txt"} Ah gotcha, missed that. Yeah that will work :-) Thanks for saying it again.

Re: How can we customise jenkins email content?

> Have you tried adding a config file through > https://builds.apache.org/job/JOB_NAME/configfiles/ ? Hi Matt, that must be from a plugin as I can't find any such url under the jobs in ci-cassandra.apache.org Otherwise there is value in having it scripted, though i did not wish to delve into xs

What's the git-websites equivalent for cloudbees (ci-cassandra.a.o)?

I'm trying to figure out how to git push, using the jenkins user, a website build on ci-cassandra.apache.org. The credentials for the jenkins user, IIUC, are normally provided by the git-websites label (agent). In Jenkins I see the jenkins (master pub key) credentials available. But even selected

Re: What's the git-websites equivalent for cloudbees (ci-cassandra.a.o)?

> To update this situation, I have just created a new 'floating' agent that > any of the Masters can attach to and use > for website build & deploy. Use the 'git-websites' label as usual. Brilliant!! This is working already at https://ci-cassandra.apache.org/job/cassandra-website/

Re: [PROPOSAL] - Change the Descriptions to installed packages on Jenkins

> This is in response to https://issues.apache.org/jira/browse/INFRA-20858 > but has also been mentioned previously over the years. +1, obvious as that's my ticket :-) Thanks for raising it for discussion Gavin. Agree with Allen's comment.

Re: [PROPOSAL] - Change the Descriptions to installed packages on Jenkins

> > > Ok, I'm done for now, please test everyone. > > Is this to have applied to ci-cassandra.apache.org (and Beam, CouchDB, Hadoop) too? Or they scheduled for later?

Lots of jenkins agents OOM'd between 22:28:58 and 22:40:44 (Tuesday 8th March)

We had scores of builds fail in this ~12 minute period. All were docker containers suddenly killed, and enough pointing towards underlying OOMs. It happened to lots of agents from various donors. Did something happen to ci-cassandra.a.o or CloudBees in this timeframe? regards, Mick

Better stability with docker authenticated jenkins agents

tl;dr Can and should all jenkins agents be (automatically) docker authenticated, for improved stability around docker commands? This past week the ci-cassandra.apache.org CI fell over because a fair percentage of docker pulls failed. Our pipeline runs a lot of docker containers. In the past week

Re: Better stability with docker authenticated jenkins agents

> > > Has anyone experience with such issues before? > > Yes. See https://issues.apache.org/jira/browse/INFRA-20795 for detail. > > In short, once Infra agrees to create the images for us, we'll move all > our CI dependencies into those containers, and should no longer have > issues. > Thanks Joa

Re: Better stability with docker authenticated jenkins agents

> > My understanding is that pulls of all images from the apache/* namespace > are not subject to rate limiting. Thus, the recommendation to move > everything you need inside of it. > As >95% of our CI docker commands are pulls from apache/ images, if rate-limiting is the cause of this (note that

Re: Better stability with docker authenticated jenkins agents

> Then it is easy to use our own Jenkins-stored creds > (https://urldefense.proofpoint.com/v2/url?u=https-3A__www.jenkins.io_doc_book_using_using-2Dcredentials_&d=DwIFaQ&c=adz96Xi0w1RHqtPMowiL2g&r=42Z7FyMoAS1DbvgKNjU8zxi7xTPVAGalPzk7bfmRVgw&m=Oa7qm3rmqlXgK8wUUTEnADpIgZT18tE6W-ghG1QUspU&s=vRKVrs-iZa

Re: docker login

> > We provide options for projects who wish to donate their own equipment for > exclusive use. > Chris, ci-cassandra.a.o keeps dockerhub credentials in jenkins credentials. These credentials are only for `docker login` to increase the pull rate limit, which without our builds will fail. Those

Re: new jenkins websites node

> We added a new websites build node which is running Ubuntu 22.04. Please > let us know if you run into any problems building on websites3. > Yup, INFRA-25207 Thanks, it will be nice with the extra agent.