Re: [CI] What are the troubles projects face with CI and Infra

2020-02-03 Thread David Nalley
rojects were feeling the pain - they had optimized their CI builds to execute really fast in Travis - essentially concurrently consuming every builder. But the reality is that some projects need more resources than others and allocating resources appropriately becomes quite the challenge. > Thank

Re: [CI] What are the troubles projects face with CI and Infra

2020-02-03 Thread David Nalley
se VMs, is > there a viable workflow? > > On 2/3/20, 6:38 PM, "David Nalley" wrote: > > Hi Alex, > > So this was explored. It creates some problems - first double the > administration overhead - most of that is automated, but it means that > our AP

Re: [CI] What are the troubles projects face with CI and Infra

2020-02-03 Thread David Nalley
and have a chance to see about the source of changes > and flag anything questionable. > > Regards, > Dave > > > On Feb 3, 2020, at 6:37 PM, David Nalley wrote: > > > > Hi Alex, > > > > So this was explored. It creates some problems - first double the >

Re: [CI] What are the troubles projects face with CI and Infra

2020-02-03 Thread David Nalley
Hi Alex, So this was explored. It creates some problems - first double the administration overhead - most of that is automated, but it means that our API usage doubles, and we're already hitting limits from Github. Second - at least one CI vendor thanked us for not doing that exactly - because th

Re: External CI Service Limitations

2019-07-18 Thread David Nalley
On Tue, Jul 16, 2019 at 5:37 PM Greg Stein wrote: > > Ray, > > Thanks for the offer of 50k minutes/project. That will definitely work for > most projects. > > While we don't have precise measurements, some projects used *way* more > than that within Travis last month: > > flink: 350k minutes > arr

Re: Syncope demo VM - Jenkins integration

2016-03-30 Thread David Nalley
On Thu, Mar 24, 2016 at 10:09 AM, andrea wrote: > > > Il 24/03/2016 16:58, John Sirois ha scritto: >> >> On Thu, Mar 24, 2016 at 9:54 AM, andrea >> wrote: >> >>> Good morning, >>> >>> I'm an Apache Syncope committer. With my team and with the kind help of >>> ASF we setup a demo machine at syncop

Re: Self-Adding Jenkins Build Machines

2016-03-30 Thread David Nalley
gt; Nikhil > > -Original Message- > From: Dmitry Blotsky [mailto:dblot...@microsoft.com] > Sent: Tuesday, March 15, 2016 6:09 PM > To: builds@apache.org > Cc: David Nalley ; jfarr...@apache.org; Daniel Takamori > > Subject: RE: Self-Adding Jenkins Build Machines &g

Re: Self-Adding Jenkins Build Machines

2016-03-07 Thread David Nalley
Hi Dmitry, builds.a.o (like ci.a.o) is a relatively generic resources - we have ~20 Jenkins nodes and that is around to serve all of the ASF. In the past, we've created project-specific VMs or physical machines when they've had unusual requirements. That's been pretty mixed though - in some cases

Re: how to get an account for "Jenkins continuous-integration server"

2015-11-30 Thread David Nalley
- ComDev Please see: http://wiki.apache.org/general/Jenkins#How_do_I_get_an_account On Fri, Nov 27, 2015 at 1:11 AM, Michael Wu wrote: > Hi Mentors, > > I'm a member from Apache Eagle project, and would like to subscribe my > account to have the permission to login "Jenkins continuous-integrat

Re: Build slave capacity on builds.apache.org

2015-08-20 Thread David Nalley
On Thu, Aug 20, 2015 at 2:55 AM, Daan Hoogland wrote: > cloudstack...! we started making more intensive use of pull-builders. the > old pull-request build job is replaced by a rat and an analysis job. Maybe > others have done so as well but I am sure we (ACS) are a culprit in this. > I thought a

Re: Build slave capacity on builds.apache.org

2015-08-20 Thread David Nalley
On Thu, Aug 20, 2015 at 6:32 AM, Gavin McDonald wrote: > >> On 20 Aug 2015, at 2:18 am, David Nalley wrote: >> >> > >> ...how many additional >> build nodes/executors will satiate our current demand for capacity? > > I know this was not aimed at me bu

Re: Name resolution issue on ubuntu jenkins slaves?

2015-08-19 Thread David Nalley
Whatever we are using for DNS on those build slaves is redirecting any NXDOMAIN to a Level3 search domain. Is this happening on the ubuntu-* slaves or on the dynamic build slaves? --David On Mon, Aug 10, 2015 at 8:59 AM, Keith W wrote: > Hello Apache builds, > > We (Apache Qpid) have been seein

Re: Build slave capacity on builds.apache.org

2015-08-19 Thread David Nalley
ck in demand - what's changed? --David On Wed, Aug 19, 2015 at 9:18 PM, David Nalley wrote: > Andrew: > > I know Jenkins tracks how big the queue is, is that something that we > can graph over time? It'd be interesting to know how that's changed > and will change, oth

Re: Build slave capacity on builds.apache.org

2015-08-19 Thread David Nalley
Andrew: I know Jenkins tracks how big the queue is, is that something that we can graph over time? It'd be interesting to know how that's changed and will change, otherwise we'll constantly be fighting fires here. I'd like to have the following items graphed: # of dynamic slaves in operation # o

Re: Potential Power loaner hardware for CI purposes

2015-07-20 Thread David Nalley
, allocate a quota to us, and just provide us with API access rather than us having to manage hardware and manage a new arch (a cost that OSUOSL staffers have already paid), we can then consume that compute capacity via the API. On Tue, Jun 16, 2015 at 3:54 PM, David Nalley wrote: > I'll need

Re: Potential Power loaner hardware for CI purposes

2015-06-16 Thread David Nalley
I'll need to reach out to OSU and see what we have available in terms of cooling/power available (frankly I don't know that we have much left). Hadoop sounds like an interested party. On Mon, Jun 15, 2015 at 3:48 PM, Sam Ruby wrote: > IBM has loaner hardware that could be targeted towards that

[jira] [Closed] (BUILDS-81) marvin build for cloudstack troubleshooting

2015-05-27 Thread David Nalley (JIRA)
[ https://issues.apache.org/jira/browse/BUILDS-81?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Nalley closed BUILDS-81. -- Resolution: Fixed abayer fixed this. Problem was due to a problem with a specific version of java

[jira] [Closed] (BUILDS-26) Install openssl-devel package on Jenkins machines

2015-05-27 Thread David Nalley (JIRA)
[ https://issues.apache.org/jira/browse/BUILDS-26?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Nalley closed BUILDS-26. -- Resolution: Fixed > Install openssl-devel package on Jenkins machi

Additional Travis-CI Capacity

2015-04-15 Thread David Nalley
FYI: https://blogs.apache.org/infra/entry/apache_gains_additional_travis_ci

Re: Access rights on Jenkins

2015-04-15 Thread David Nalley
Yes. As a general principle we try and minimize the access that folks have in general to external-facing systems. Jenkins is one of several CI platforms that the ASF makes use of, and so it doesn't make sense to hand out access needlessly to the ~5000 committers, of which only 543 have requested ac

[jira] [Commented] (BUILDS-39) Test Infrastructure for Usergrid podling

2014-12-10 Thread David Nalley (JIRA)
[ https://issues.apache.org/jira/browse/BUILDS-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14241211#comment-14241211 ] David Nalley commented on BUILDS-39: Just a couple of comments. We can provide

Re: Automatic builds by Jenkins for pull requests to github

2014-12-04 Thread David Nalley
Hi Taegeon, Have you seen this and followed those steps: https://blogs.apache.org/infra/entry/github_pull_request_builds_now --David On Wed, Dec 3, 2014 at 3:50 AM, Tae-Geon Um wrote: > Hi all, > > I’m trying to connect Jenkins CI to Apache REEF, an incubator project I am > working on. I’d lik

[jira] [Resolved] (BUILDS-33) Jenkins to GIT fetch timeouts (Jena)

2014-12-01 Thread David Nalley (JIRA)
[ https://issues.apache.org/jira/browse/BUILDS-33?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Nalley resolved BUILDS-33. Resolution: Fixed Andy: I think this was resolved by the work done to reinstantiate the git.a.o

Re: npm credentials on jenkins

2014-11-26 Thread David Nalley
On Wed, Nov 26, 2014 at 2:04 PM, Steven Gill wrote: > Thanks for responding David! > > Just to clarify, we don't consider nightlys releases. They are just > snapshots and are published under the nightly tag to npm. You need to be > explicit if you want to download it. It is only meant for testing.

Re: npm credentials on jenkins

2014-11-26 Thread David Nalley
On Tue, Nov 25, 2014 at 4:48 PM, Steven Gill wrote: > Hey Everyone, > > So I need to add credentials to my workspace for uploading a nightly build > to npm for cordova. > > The command I need to run is `npm login` or `npm addUser`. It would prompt > me to add my username, password and email. I thi

[jira] [Updated] (BUILDS-31) Jenkins builds fail on H10: java.lang.OutOfMemoryError: GC overhead limit exceeded

2014-11-18 Thread David Nalley (JIRA)
[ https://issues.apache.org/jira/browse/BUILDS-31?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Nalley updated BUILDS-31: --- Assignee: Andrew Bayer > Jenkins builds fail on H10: java.lang.OutOfMemoryError: GC overhead li

[jira] [Updated] (BUILDS-30) More recent version of JDK6 including strong encryption on Ubuntu slaves

2014-10-30 Thread David Nalley (JIRA)
[ https://issues.apache.org/jira/browse/BUILDS-30?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Nalley updated BUILDS-30: --- Assignee: Andrew Bayer > More recent version of JDK6 including strong encryption on Ubuntu sla

[jira] [Commented] (BUILDS-23) Failed to deploy artifacts: Could not transfer artifact on jenkins-ubuntu-1404-4gb-e0d

2014-10-01 Thread David Nalley (JIRA)
[ https://issues.apache.org/jira/browse/BUILDS-23?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1412#comment-1412 ] David Nalley commented on BUILDS-23: [~abayer] take a look here: h

Ulimit on machines

2014-10-01 Thread David Nalley
Rajiv: Can you take a look at BUILDS-17 - we don't seem to be able to change ulimit on a number of machines at Yahoo!. Or rather, it's not preserved past reboot it seems. https://issues.apache.org/jira/browse/BUILDS-17 Thanks, --David

Re: NOTICE: New Rackspace on-demand slaves added to builds.a.o

2014-08-26 Thread David Nalley
On Tue, Aug 26, 2014 at 2:14 PM, Roman Shaposhnik wrote: > On Tue, Aug 26, 2014 at 10:51 AM, David Nalley wrote: >> So a couple of comments. >> 1. Would Cloudera consider sponsoring/allowing us to consume some of >> those EC2 credits? :) I think Dan's work on image gen

Re: NOTICE: New Rackspace on-demand slaves added to builds.a.o

2014-08-26 Thread David Nalley
On Tue, Aug 26, 2014 at 12:42 PM, Roman Shaposhnik wrote: > On Sat, Aug 23, 2014 at 6:32 PM, David Nalley wrote: >> These slaves are on-demand. If they aren't setup properly to begin >> with, a lot of the value is lost. > > Question: is there any chance we can start

Re: NOTICE: New Rackspace on-demand slaves added to builds.a.o

2014-08-23 Thread David Nalley
These slaves are on-demand. If they aren't setup properly to begin with, a lot of the value is lost. If it takes us 30 minutes for an on-demand slave to be ready to do any work, especially if it was to run a 3 minute job, there was no point in the first place. That said, most of the work done for

Re: The apache build server

2014-08-21 Thread David Nalley
Hi Alex, et al: We've got a few other things in the wings to work on that problem as well. Both Azure and Rackspace have donated cloud credits to be used for build slaves. The work to have dynamic build slaves available from Rackspace is almost done, which means as the number of jobs in the queue

Re: Replacement Jenkins Slaves Online. Old Slaves going offline next week

2014-07-16 Thread David Nalley
On Wed, Jul 16, 2014 at 8:40 PM, Andrew Bayer wrote: > Yeah, I'd really like to see where the ansible scripts are and how to > run them on slaves, since I'm babysitting builds.a.o a lot of the > time. =) > > A. > https://github.com/apache/toolchain --David

Re: Buildbot builds on bb-vm_ubuntu - No space left on device

2014-07-11 Thread David Nalley
I think Jake tried to clear up some space intermittently. On Fri, Jul 11, 2014 at 8:12 AM, Gavin McDonald wrote: > > On 10/07/2014, at 9:27 AM, Konstantin Kolinko wrote: > >> 2014-07-07 20:59 GMT+04:00 Konstantin Kolinko : >>> 2014-07-03 17:34 GMT+04:00 Konstantin Kolinko : Hi! Bu

Re: Latest status on the hanging Jenkins slaves/builds

2014-05-23 Thread David Nalley
Does it make sense to start with disabling all maven plugin-based jobs first to see if that helps with the hanging issue?* Purging 600 jobs seems like a rash first step if we aren't certain that it is indeed the problem. Obviously this will need lots of visibility prior to doing it, notice on buil

Re: Heads up - Planning to disable anonymous workspace read access on builds.a.o

2014-05-16 Thread David Nalley
On Thu, May 15, 2014 at 4:55 PM, Andrew Bayer wrote: > So from https://issues.jenkins-ci.org/browse/JENKINS-23056 it sounds like at > least one of the problems we're having with Jenkins hanging is because of > attempts to access the workspace of jobs through the UI - when a slave is > slow or hang

Re: Buildbot unavailable

2014-04-20 Thread David Nalley
On Sat, Apr 19, 2014 at 3:38 PM, Dennis Lundberg wrote: > Hi, > > I've been trying to reach this Buildbot URL for a couple of hours now, > but it doesnt seem to be available: > http://ci.apache.org/builders/maven-site-staging > > -- > Dennis Lundberg It's a known issue: IIRC its waiting on disks

Re: Windows slaves (1 and 2) offline

2014-04-07 Thread David Nalley
Jan and I discussed this briefly at ApacheCon and are tossing around the idea of having Circonus monitor the status of the slave (according to Jenkins) and perhaps to take corrective action automagically. We're going to continue to think and work on this. Neither of us have admin privs on the Windo

Re: volunteering for ASF Jenkins farm service maintenance

2014-04-03 Thread David Nalley
On Thu, Apr 3, 2014 at 4:48 PM, OmPrakash Muppirala wrote: > On Thu, Apr 3, 2014 at 1:38 PM, Jake Farrell wrote: > >> Om, Herve >> What action are you specifically trying to achieve with the role you are >> referring to as 'maintainer' ? >> > > Things like setting up builds which require more tha

Re: FATAL: Could not apply tag with git-jenkins Any23-trunk job.

2013-06-09 Thread David Nalley
On Sun, Jun 9, 2013 at 8:31 AM, Olivier Lamy wrote: > 2013/6/9 David Nalley : >> On Sun, Jun 9, 2013 at 6:27 AM, Olivier Lamy wrote: >>> In your job configuration in advanced git section: check the option >>> called "Skip internal tag" >>> That

Re: FATAL: Could not apply tag with git-jenkins Any23-trunk job.

2013-06-09 Thread David Nalley
On Sun, Jun 9, 2013 at 6:27 AM, Olivier Lamy wrote: > In your job configuration in advanced git section: check the option > called "Skip internal tag" > That must fix that normally > Something has changed in the config - as this has been enabled across what appears to be all of the cloudstack job

Re: Publican on an ubuntuX slave?

2013-01-08 Thread David Nalley
On Thu, Dec 20, 2012 at 4:31 PM, Niklas Gustavsson wrote: > On Fri, Dec 14, 2012 at 6:55 AM, David Nalley wrote: > >> Any chance I can have publican installed on one of the Ubuntu 12.04 slaves? >> > > Installed on vesta.a.o (ubuntu2). > Niklas: Thanks for picking

Publican on an ubuntuX slave?

2012-12-13 Thread David Nalley
Hi folks: Any chance I can have publican installed on one of the Ubuntu 12.04 slaves? CloudStack uses publican to validate/test/build its DocBook XML-based documentation. --David