Number of build prozessors on Lucene node
Hi, could someone please change back the number of "build processors" for the "lucene" Jenkins node to 1? Currently it executes always 2 Jobs in parallel. The underlying server only has 4 cpu cores and the Lucene Job configuration is done in a way that it uses all available CPU nodes, so running 2 builds in parallel on is in most cases not a good idea. There are in fact some jobs, that don't require much CPU or are not multithreaded (like artifact builds), but those are generally quick. The main tasks taking several hours to execute are very CPU intensive - the reason why we have an own slave. Any information why this was changed? Unfortunately I don't have the power to configure this correctly. Uwe - Uwe Schindler uschind...@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/
Re: Number of build prozessors on Lucene node
> On 20 Aug 2015, at 9:17 am, Uwe Schindler wrote: > > Hi, > > could someone please change back the number of "build processors" for the > "lucene" Jenkins node to 1? Currently it executes always 2 Jobs in parallel. > The underlying server only has 4 cpu cores and the Lucene Job configuration > is done in a way that it uses all available CPU nodes, so running 2 builds in > parallel on is in most cases not a good idea. There are in fact some jobs, > that don't require much CPU or are not multithreaded (like artifact builds), > but those are generally quick. The main tasks taking several hours to execute > are very CPU intensive - the reason why we have an own slave. > > Any information why this was changed? Unfortunately I don't have the power to > configure this correctly. I changed it short term to help with backlog. When I did this (2 days ago) there were 166 builds in the queue, over 30 of those were waiting on the Lucene node. Will change it back now. HTH Gav… > > Uwe > signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Number of build prozessors on Lucene node
Hi, The backlog of Lucene is generally large. That's wanted, because we don't trigger builds based on commits; just to always have one build of everything in the queue we do it time-based. Because of our pseudorandomized test infrastructure, the build slave should never be out of work because every run may trigger a bug in Lucene or the Java VM. Because Jenkins never triggers the same job multiple times, it's perfectly fine to have at least one job of all in the queue. If you don't want your statistics broken, you may exclude the lucene slave from counting towards queue size. It's completely on its own and only allows ecplicitely assigned jobs. You can always remove triggered builds from queu (I sometimes do this for testing puposes or to trigger a special build manually). They get queued anyways a while later if deleted. Uwe Am 20. August 2015 11:15:17 MESZ, schrieb Gavin McDonald : > >> On 20 Aug 2015, at 9:17 am, Uwe Schindler >wrote: >> >> Hi, >> >> could someone please change back the number of "build processors" for >the "lucene" Jenkins node to 1? Currently it executes always 2 Jobs in >parallel. The underlying server only has 4 cpu cores and the Lucene Job >configuration is done in a way that it uses all available CPU nodes, so >running 2 builds in parallel on is in most cases not a good idea. There >are in fact some jobs, that don't require much CPU or are not >multithreaded (like artifact builds), but those are generally quick. >The main tasks taking several hours to execute are very CPU intensive - >the reason why we have an own slave. >> >> Any information why this was changed? Unfortunately I don't have the >power to configure this correctly. > >I changed it short term to help with backlog. >When I did this (2 days ago) there were 166 builds in the queue, over >30 of those were waiting on the Lucene node. > >Will change it back now. > >HTH > >Gav… > >> >> Uwe >> -- Uwe Schindler H.-H.-Meier-Allee 63, 28213 Bremen http://www.thetaphi.de
Re: Number of build prozessors on Lucene node
> On 20 Aug 2015, at 10:28 am, Uwe Schindler wrote: > > Hi, > > The backlog of Lucene is generally large. That's wanted, No problem, thanks for letting me know, I’ll take that into account in the future. > because we don't trigger builds based on commits; just to always have one > build of everything in the queue we do it time-based. Because of our > pseudorandomized test infrastructure, the build slave should never be out of > work because every run may trigger a bug in Lucene or the Java VM. Because > Jenkins never triggers the same job multiple times, it's perfectly fine to > have at least one job of all in the queue. > > If you don't want your statistics broken, I’m more concerned about the health of the services we provide and what to do about them when they need fixing and|or improvement. Stats are nice, they provide feedback for current and future improvements that can be made, but I’m not in the habit of twisting stats just to make things look pretty and good, thats not my take on what stats are for. Gav… > you may exclude the lucene slave from counting towards queue size. It's > completely on its own and only allows ecplicitely assigned jobs. > > You can always remove triggered builds from queu (I sometimes do this for > testing puposes or to trigger a special build manually). They get queued > anyways a while later if deleted. > > Uwe > > Am 20. August 2015 11:15:17 MESZ, schrieb Gavin McDonald > : >> >>> On 20 Aug 2015, at 9:17 am, Uwe Schindler >> wrote: >>> >>> Hi, >>> >>> could someone please change back the number of "build processors" for >> the "lucene" Jenkins node to 1? Currently it executes always 2 Jobs in >> parallel. The underlying server only has 4 cpu cores and the Lucene Job >> configuration is done in a way that it uses all available CPU nodes, so >> running 2 builds in parallel on is in most cases not a good idea. There >> are in fact some jobs, that don't require much CPU or are not >> multithreaded (like artifact builds), but those are generally quick. >> The main tasks taking several hours to execute are very CPU intensive - >> the reason why we have an own slave. >>> >>> Any information why this was changed? Unfortunately I don't have the >> power to configure this correctly. >> >> I changed it short term to help with backlog. >> When I did this (2 days ago) there were 166 builds in the queue, over >> 30 of those were waiting on the Lucene node. >> >> Will change it back now. >> >> HTH >> >> Gav… >> >>> >>> Uwe >>> > > -- > Uwe Schindler > H.-H.-Meier-Allee 63, 28213 Bremen > http://www.thetaphi.de Gav... (( ( ( )\ ) )\ ) )\ ) ( )) )\(()/((()/( (()/( )\ ) ( ) ( /( ( (( /( ( ( ( _)( /(_))/(_)) /(_)) ( (()/( )( ( /( ( )\()))())\ ( )\()) ))\ )())\ )\ _ )\ (_)) (_))_| (_)) )\ ) /(_))(()\ )(_)) )\ (_))/(()\ /((_) )\ (_))/ /((_)(()\ /((_) (_)_\(_)/ __|| |_ |_ _| _(_/((_) _| ((_)((_)_ ((_)| |_ ((_)(_))( ((_)| |_ (_))( ((_)(_)) / _ \ \__ \| __| | | | ' \))| _|| '_|/ _` |(_-<| _|| '_|| || |/ _| | _|| || || '_|/ -_) /_/ \_\ |___/|_||___||_||_| |_| |_| \__,_|/__/ \__||_| \_,_|\__| \__| \_,_||_| \___|
Re: Build slave capacity on builds.apache.org
> On 20 Aug 2015, at 7:55 am, Daan Hoogland wrote: > > cloudstack...! we started making more intensive use of pull-builders. the > old pull-request build job is replaced by a rat and an analysis job. Maybe > others have done so as well but I am sure we (ACS) are a culprit in this. From what I can tell the Cloustack builds are not targeting the dynamic slaves on purpose but are using the generic ‘ubuntu’ label. So what is happening is that the ubuntu-* machines are more or less in constant use so the dynamic slaves are getting triggered more and more. The dynamic slaves have the ‘ubuntu’ label also so this is why. To reduce the use of the dynamic slaves and therefore the costs involved in having these on more and more is to increase our pool of static slaves. Gav… > > On Thu, Aug 20, 2015 at 4:28 AM, David Nalley wrote: > >> So, just spot checking some of the dynamic build slaves - historically >> those have spun up a few hours at a time, to give us additional >> capacity when our queue spiked - but it looks like the slaves are >> staying online pretty much all of the time. Right now we are >> configured to have 12 slaves. Assuming they are staying online all the >> time, that's roughly $2600 per month that we are spending on build >> slave capacity - I looked at our last bill ending July 24th, and we >> spent about $1200 on build slave capacity - so this is about 2x what >> we were spending - I am curious why the sudden uptick in demand - >> what's changed? >> >> --David >> >> On Wed, Aug 19, 2015 at 9:18 PM, David Nalley wrote: >>> Andrew: >>> >>> I know Jenkins tracks how big the queue is, is that something that we >>> can graph over time? It'd be interesting to know how that's changed >>> and will change, otherwise we'll constantly be fighting fires here. >>> >>> I'd like to have the following items graphed: >>> >>> # of dynamic slaves in operation >>> # of executors currently in use >>> # of jobs in the queue >>> # of jobs in the queue actually waiting on an executor. >>> >>> It'd be nice to know the average time waiting on an executor as well. >>> >>> >>> I don't suppose Jenkins has any functionality that would allow us to >>> run a quota per-project is there? >>> >>> Since you're one of the resident Jenkins experts how many additional >>> build nodes/executors will satiate our current demand for capacity? >>> >>> >>> --David >>> >>> >>> On Tue, Aug 18, 2015 at 11:16 AM, Andrew Bayer >> wrote: Hey all - So as you may have noticed, we've seen an increase in build utilization >> on builds.a.o in the last few months - which is great! The problem is that we're seeing more demand than we have resources at this point, and >> that's only going to increase. We're working on getting more slaves lined up, >> but don't yet have a timeframe for that - we'll let you all know when we >> have more news. A. >> > > > > -- > Daan Gav... (( ( ( )\ ) )\ ) )\ ) ( )) )\(()/((()/( (()/( )\ ) ( ) ( /( ( (( /( ( ( ( _)( /(_))/(_)) /(_)) ( (()/( )( ( /( ( )\()))())\ ( )\()) ))\ )())\ )\ _ )\ (_)) (_))_| (_)) )\ ) /(_))(()\ )(_)) )\ (_))/(()\ /((_) )\ (_))/ /((_)(()\ /((_) (_)_\(_)/ __|| |_ |_ _| _(_/((_) _| ((_)((_)_ ((_)| |_ ((_)(_))( ((_)| |_ (_))( ((_)(_)) / _ \ \__ \| __| | | | ' \))| _|| '_|/ _` |(_-<| _|| '_|| || |/ _| | _|| || || '_|/ -_) /_/ \_\ |___/|_||___||_||_| |_| |_| \__,_|/__/ \__||_| \_,_|\__| \__| \_,_||_| \___|
Re: Name resolution issue on ubuntu jenkins slaves?
> On 20 Aug 2015, at 4:03 am, David Nalley wrote: > > Whatever we are using for DNS on those build slaves is redirecting any > NXDOMAIN to a Level3 search domain. > > Is this happening on the ubuntu-* slaves or on the dynamic build slaves? It was mentioned earlier : > I am certainly seeing this on slaves ubuntu4 > through ubuntu6. I have seen passes and failures on the dynamic slaves too, so seems inconsistent. Gav… > > --David > > On Mon, Aug 10, 2015 at 8:59 AM, Keith W wrote: >> Hello Apache builds, >> >> We (Apache Qpid) have been seeing a test fail on the Ubuntu Jenkins slaves >> since August 9th. The test verifies the behaviour of the code when the >> user specifies a hostname that does not exist in DNS. For this purpose, >> the test uses a random name 'hg3sgaaw4lgihjs' (without hierarchal part) >> which is assumed not resolve. This test is longstanding and has been >> running on the slaves for many years without issue. >> >> Between August 9th and 10th, something appears to have changed on the >> slaves, which is meaning that the lookup of the name is now returning an >> IP. This is causing the Java test to fail. I've investigated by >> introducing shell commands into the job, and can see evidence of the same >> problem at the UNIX level. I am certainly seeing this on slaves ubuntu4 >> through ubuntu6. >> >> >> $ host hg3sgaaw4lgihjs >> hg3sgaaw4lgihjs has address 198.105.244.11 >> hg3sgaaw4lgihjs has address 198.105.254.11 >> Host hg3sgaaw4lgihjs not found: 3(NXDOMAIN) >> >> $ host hg3sgaaw4lgihjs2 >> hg3sgaaw4lgihjs2 has address 198.105.244.11 >> hg3sgaaw4lgihjs2 has address 198.105.254.11 >> Host hg3sgaaw4lgihjs2 not found: 3(NXDOMAIN) >> >> >> I considered changing the test to use a RFC-2606 Reserved Top Level DNS >> Names hg3sgaaw4lgihjs.invalid but I notice that it too is resolving to >> 198.105.244.11 too. The fact that an .invalid address is resolving makes >> me suspect there is an environmental problem at the root cause. >> >> host hg3sgaaw4lgihjs.invalid >> hg3sgaaw4lgihjs.invalid has address 198.105.244.11 >> hg3sgaaw4lgihjs.invalid has address 198.105.254.11 >> Host hg3sgaaw4lgihjs.invalid not found: 3(NXDOMAIN) >> >> Is there a name resolution issue affecting these hosts? >> >> Example job affected by the issue: >> >> https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-Test-JDK1.8/ >> >> Kind regards, Keith Wall. signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Build slave capacity on builds.apache.org
> On 20 Aug 2015, at 3:28 am, David Nalley wrote: > > So, just spot checking some of the dynamic build slaves - historically > those have spun up a few hours at a time, to give us additional > capacity when our queue spiked - but it looks like the slaves are > staying online pretty much all of the time. Right now we are > configured to have 12 slaves. Assuming they are staying online all the > time, that's roughly $2600 per month that we are spending on build > slave capacity - I looked at our last bill ending July 24th, and we > spent about $1200 on build slave capacity - so this is about 2x what > we were spending - I am curious why the sudden uptick in demand - > what's changed? We have seen more projects starting to use jenkins and also existing projects are increasing their tests. The demand for the dynamic slaves seems to be the availability of the static slaves are getting less and less so dynamic slaves are being spun up much more often. So no projects that I can see are targeting the dynamic slaves specifically. Gav… > > --David > > On Wed, Aug 19, 2015 at 9:18 PM, David Nalley wrote: >> Andrew: >> >> I know Jenkins tracks how big the queue is, is that something that we >> can graph over time? It'd be interesting to know how that's changed >> and will change, otherwise we'll constantly be fighting fires here. >> >> I'd like to have the following items graphed: >> >> # of dynamic slaves in operation >> # of executors currently in use >> # of jobs in the queue >> # of jobs in the queue actually waiting on an executor. >> >> It'd be nice to know the average time waiting on an executor as well. >> >> >> I don't suppose Jenkins has any functionality that would allow us to >> run a quota per-project is there? >> >> Since you're one of the resident Jenkins experts how many additional >> build nodes/executors will satiate our current demand for capacity? >> >> >> --David >> >> >> On Tue, Aug 18, 2015 at 11:16 AM, Andrew Bayer >> wrote: >>> Hey all - >>> >>> So as you may have noticed, we've seen an increase in build utilization on >>> builds.a.o in the last few months - which is great! The problem is that >>> we're seeing more demand than we have resources at this point, and that's >>> only going to increase. We're working on getting more slaves lined up, but >>> don't yet have a timeframe for that - we'll let you all know when we have >>> more news. >>> >>> A. Gav... (( ( ( )\ ) )\ ) )\ ) ( )) )\(()/((()/( (()/( )\ ) ( ) ( /( ( (( /( ( ( ( _)( /(_))/(_)) /(_)) ( (()/( )( ( /( ( )\()))())\ ( )\()) ))\ )())\ )\ _ )\ (_)) (_))_| (_)) )\ ) /(_))(()\ )(_)) )\ (_))/(()\ /((_) )\ (_))/ /((_)(()\ /((_) (_)_\(_)/ __|| |_ |_ _| _(_/((_) _| ((_)((_)_ ((_)| |_ ((_)(_))( ((_)| |_ (_))( ((_)(_)) / _ \ \__ \| __| | | | ' \))| _|| '_|/ _` |(_-<| _|| '_|| || |/ _| | _|| || || '_|/ -_) /_/ \_\ |___/|_||___||_||_| |_| |_| \__,_|/__/ \__||_| \_,_|\__| \__| \_,_||_| \___|
Re: Build slave capacity on builds.apache.org
> On 20 Aug 2015, at 2:18 am, David Nalley wrote: > > > ...how many additional > build nodes/executors will satiate our current demand for capacity? I know this was not aimed at me but I’ll give my opinion. From what I’ve seen of the current growth over the last year; and to allow for future expansion over the next 2 years I would look to get another 20 static slave machines/vms , each running 2 executors. Gav… > > > --David > > > On Tue, Aug 18, 2015 at 11:16 AM, Andrew Bayer wrote: >> Hey all - >> >> So as you may have noticed, we've seen an increase in build utilization on >> builds.a.o in the last few months - which is great! The problem is that >> we're seeing more demand than we have resources at this point, and that's >> only going to increase. We're working on getting more slaves lined up, but >> don't yet have a timeframe for that - we'll let you all know when we have >> more news. >> >> A. signature.asc Description: Message signed with OpenPGP using GPGMail
[jira] [Commented] (BUILDS-106) update Maven version for Maven builds
[ https://issues.apache.org/jira/browse/BUILDS-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704738#comment-14704738 ] Gavin commented on BUILDS-106: -- maven-archetype: maven 3.0.3 -> maven 3.2.2 :: jdk 1.6.0_24 -> no change :: mailto set to notificati...@maven.apache.org ran a build and it failed: [WARNING] The requested profile "apache.snapshots" could not be activated because it does not exist. [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.11:check (rat-check) on project maven-archetype: Too many files with unapproved license: 1 See RAT report in: /x1/jenkins/jenkins-master/jenkins-home/jobs/maven-archetype/workspace/trunk/target/rat.txt -> [Help 1] > update Maven version for Maven builds > - > > Key: BUILDS-106 > URL: https://issues.apache.org/jira/browse/BUILDS-106 > Project: Infra Build Platform > Issue Type: Task > Components: Analysis >Reporter: Hervé Boutemy > > it seems a good number of jobs for Maven run with Maven [3.0,3.0.3], which > fails the build: > here is the list: https://analysis.apache.org/jenkins/view/maven/ > and one faiklure example: > https://analysis.apache.org/jenkins/view/maven/job/maven-parent-pom/113/console > Please update Maven version used to a newer one (3.0.5 or more) > notice: if build failures could be sent to notificati...@maven.apache.org, > please, this would ease detection of such failing analysis builds by Maven > team -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (BUILDS-106) update Maven version for Maven builds
[ https://issues.apache.org/jira/browse/BUILDS-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704745#comment-14704745 ] Gavin edited comment on BUILDS-106 at 8/20/15 12:15 PM: maven-doxia: maven 3.0.4 -> maven 3.2.2 :: jdk 1.6.0_45 -> no change :: mailto set to notificati...@maven.apache.org ran a build and it failed: ERROR: No such file /x1/jenkins/jenkins-master/jenkins-home/jobs/maven-doxia/workspace/pom.xml Perhaps you need to specify the correct POM file path in the project configuration? Skipping sonar analysis due to bad build status FAILURE IRC notifier plugin: Sending notification to: #asftest IRC notifier plugin: [ERROR] not connected. Cannot send message to '#asftest' was (Author: ipv6guru): maven-doxia: maven 3.0.4 -> maven 3.2.2 :: jdk 1.1.0_45 -> no change :: mailto set to notificati...@maven.apache.org ran a build and it failed: ERROR: No such file /x1/jenkins/jenkins-master/jenkins-home/jobs/maven-doxia/workspace/pom.xml Perhaps you need to specify the correct POM file path in the project configuration? Skipping sonar analysis due to bad build status FAILURE IRC notifier plugin: Sending notification to: #asftest IRC notifier plugin: [ERROR] not connected. Cannot send message to '#asftest' > update Maven version for Maven builds > - > > Key: BUILDS-106 > URL: https://issues.apache.org/jira/browse/BUILDS-106 > Project: Infra Build Platform > Issue Type: Task > Components: Analysis >Reporter: Hervé Boutemy > > it seems a good number of jobs for Maven run with Maven [3.0,3.0.3], which > fails the build: > here is the list: https://analysis.apache.org/jenkins/view/maven/ > and one faiklure example: > https://analysis.apache.org/jenkins/view/maven/job/maven-parent-pom/113/console > Please update Maven version used to a newer one (3.0.5 or more) > notice: if build failures could be sent to notificati...@maven.apache.org, > please, this would ease detection of such failing analysis builds by Maven > team -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BUILDS-106) update Maven version for Maven builds
[ https://issues.apache.org/jira/browse/BUILDS-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704745#comment-14704745 ] Gavin commented on BUILDS-106: -- maven-doxia: maven 3.0.4 -> maven 3.2.2 :: jdk 1.1.0_45 -> no change :: mailto set to notificati...@maven.apache.org ran a build and it failed: ERROR: No such file /x1/jenkins/jenkins-master/jenkins-home/jobs/maven-doxia/workspace/pom.xml Perhaps you need to specify the correct POM file path in the project configuration? Skipping sonar analysis due to bad build status FAILURE IRC notifier plugin: Sending notification to: #asftest IRC notifier plugin: [ERROR] not connected. Cannot send message to '#asftest' > update Maven version for Maven builds > - > > Key: BUILDS-106 > URL: https://issues.apache.org/jira/browse/BUILDS-106 > Project: Infra Build Platform > Issue Type: Task > Components: Analysis >Reporter: Hervé Boutemy > > it seems a good number of jobs for Maven run with Maven [3.0,3.0.3], which > fails the build: > here is the list: https://analysis.apache.org/jenkins/view/maven/ > and one faiklure example: > https://analysis.apache.org/jenkins/view/maven/job/maven-parent-pom/113/console > Please update Maven version used to a newer one (3.0.5 or more) > notice: if build failures could be sent to notificati...@maven.apache.org, > please, this would ease detection of such failing analysis builds by Maven > team -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BUILDS-106) update Maven version for Maven builds
[ https://issues.apache.org/jira/browse/BUILDS-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704751#comment-14704751 ] Gavin commented on BUILDS-106: -- maven-parent-pom: maven 3.0.3 -> maven 3.2.2 :: jdk 1.6.0_24 -> no change :: mailto set to notificati...@maven.apache.org ran a build and it passed. > update Maven version for Maven builds > - > > Key: BUILDS-106 > URL: https://issues.apache.org/jira/browse/BUILDS-106 > Project: Infra Build Platform > Issue Type: Task > Components: Analysis >Reporter: Hervé Boutemy > > it seems a good number of jobs for Maven run with Maven [3.0,3.0.3], which > fails the build: > here is the list: https://analysis.apache.org/jenkins/view/maven/ > and one faiklure example: > https://analysis.apache.org/jenkins/view/maven/job/maven-parent-pom/113/console > Please update Maven version used to a newer one (3.0.5 or more) > notice: if build failures could be sent to notificati...@maven.apache.org, > please, this would ease detection of such failing analysis builds by Maven > team -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BUILDS-106) update Maven version for Maven builds
[ https://issues.apache.org/jira/browse/BUILDS-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704759#comment-14704759 ] Gavin commented on BUILDS-106: -- maven-plugin-tools: maven 3.0.4 -> maven 3.2.2 :: jdk 1.6.0_24 -> no change :: mailto set to notificati...@maven.apache.org ran a build and it failed:: [ERROR] Child module /x1/jenkins/jenkins-master/jenkins-home/jobs/maven-plugin-tools/workspace/maven-plugin-plugin/pom.xml of /x1/jenkins/jenkins-master/jenkins-home/jobs/maven-plugin-tools/workspace/pom.xml does not exist @ > update Maven version for Maven builds > - > > Key: BUILDS-106 > URL: https://issues.apache.org/jira/browse/BUILDS-106 > Project: Infra Build Platform > Issue Type: Task > Components: Analysis >Reporter: Hervé Boutemy > > it seems a good number of jobs for Maven run with Maven [3.0,3.0.3], which > fails the build: > here is the list: https://analysis.apache.org/jenkins/view/maven/ > and one faiklure example: > https://analysis.apache.org/jenkins/view/maven/job/maven-parent-pom/113/console > Please update Maven version used to a newer one (3.0.5 or more) > notice: if build failures could be sent to notificati...@maven.apache.org, > please, this would ease detection of such failing analysis builds by Maven > team -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BUILDS-106) update Maven version for Maven builds
[ https://issues.apache.org/jira/browse/BUILDS-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704768#comment-14704768 ] Gavin commented on BUILDS-106: -- maven-release: maven 3.0.4 -> maven 3.2.2 :: jdk 1.7.0_65 -> no change :: mailto set to notificati...@maven.apache.org ran a build and it passed. > update Maven version for Maven builds > - > > Key: BUILDS-106 > URL: https://issues.apache.org/jira/browse/BUILDS-106 > Project: Infra Build Platform > Issue Type: Task > Components: Analysis >Reporter: Hervé Boutemy > > it seems a good number of jobs for Maven run with Maven [3.0,3.0.3], which > fails the build: > here is the list: https://analysis.apache.org/jenkins/view/maven/ > and one faiklure example: > https://analysis.apache.org/jenkins/view/maven/job/maven-parent-pom/113/console > Please update Maven version used to a newer one (3.0.5 or more) > notice: if build failures could be sent to notificati...@maven.apache.org, > please, this would ease detection of such failing analysis builds by Maven > team -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BUILDS-106) update Maven version for Maven builds
[ https://issues.apache.org/jira/browse/BUILDS-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704789#comment-14704789 ] Gavin commented on BUILDS-106: -- maven-shared: maven 3.0.5 -> maven 3.2.2 :: jdk 1.7.0_51 -> no change :: mailto set to notificati...@maven.apache.org ran a build and it failed: compilation errors > update Maven version for Maven builds > - > > Key: BUILDS-106 > URL: https://issues.apache.org/jira/browse/BUILDS-106 > Project: Infra Build Platform > Issue Type: Task > Components: Analysis >Reporter: Hervé Boutemy > > it seems a good number of jobs for Maven run with Maven [3.0,3.0.3], which > fails the build: > here is the list: https://analysis.apache.org/jenkins/view/maven/ > and one faiklure example: > https://analysis.apache.org/jenkins/view/maven/job/maven-parent-pom/113/console > Please update Maven version used to a newer one (3.0.5 or more) > notice: if build failures could be sent to notificati...@maven.apache.org, > please, this would ease detection of such failing analysis builds by Maven > team -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BUILDS-106) update Maven version for Maven builds
[ https://issues.apache.org/jira/browse/BUILDS-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704807#comment-14704807 ] Gavin commented on BUILDS-106: -- maven-surefire: maven 3.0.4 -> maven 3.2.2 :: jdk 1.6.0_24 -> no change :: mailto set to notificati...@maven.apache.org ran a build and it failed: [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.11:check (rat-check) on project surefire: Too many files with unapproved license: 542 See RAT report in: /x1/jenkins/jenkins-master/jenkins-home/jobs/maven-surefire/workspace/target/rat.txt -> [Help 1] > update Maven version for Maven builds > - > > Key: BUILDS-106 > URL: https://issues.apache.org/jira/browse/BUILDS-106 > Project: Infra Build Platform > Issue Type: Task > Components: Analysis >Reporter: Hervé Boutemy > > it seems a good number of jobs for Maven run with Maven [3.0,3.0.3], which > fails the build: > here is the list: https://analysis.apache.org/jenkins/view/maven/ > and one faiklure example: > https://analysis.apache.org/jenkins/view/maven/job/maven-parent-pom/113/console > Please update Maven version used to a newer one (3.0.5 or more) > notice: if build failures could be sent to notificati...@maven.apache.org, > please, this would ease detection of such failing analysis builds by Maven > team -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BUILDS-106) update Maven version for Maven builds
[ https://issues.apache.org/jira/browse/BUILDS-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704817#comment-14704817 ] Gavin commented on BUILDS-106: -- maven-wagon: maven 3.0.4 -> maven 3.2.2 :: jdk 1.6.0_24 - no change :: mailto set to notificati...@maven.apache.org ran a build and it failed compilation errors > update Maven version for Maven builds > - > > Key: BUILDS-106 > URL: https://issues.apache.org/jira/browse/BUILDS-106 > Project: Infra Build Platform > Issue Type: Task > Components: Analysis >Reporter: Hervé Boutemy > > it seems a good number of jobs for Maven run with Maven [3.0,3.0.3], which > fails the build: > here is the list: https://analysis.apache.org/jenkins/view/maven/ > and one faiklure example: > https://analysis.apache.org/jenkins/view/maven/job/maven-parent-pom/113/console > Please update Maven version used to a newer one (3.0.5 or more) > notice: if build failures could be sent to notificati...@maven.apache.org, > please, this would ease detection of such failing analysis builds by Maven > team -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BUILDS-106) update Maven version for Maven builds
[ https://issues.apache.org/jira/browse/BUILDS-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704825#comment-14704825 ] Gavin commented on BUILDS-106: -- tomcat-maven-plugin: maven 3.1.1 -> maven 3.2.2 :: jdk 1.6.0_24 -> no change :: mailto set to notificati...@maven.apache.org ran a build and it failed: ERROR: No such file /x1/jenkins/jenkins-master/jenkins-home/jobs/tomcat-maven-plugin/workspace/pom.xml Perhaps you need to specify the correct POM file path in the project configuration? > update Maven version for Maven builds > - > > Key: BUILDS-106 > URL: https://issues.apache.org/jira/browse/BUILDS-106 > Project: Infra Build Platform > Issue Type: Task > Components: Analysis >Reporter: Hervé Boutemy > > it seems a good number of jobs for Maven run with Maven [3.0,3.0.3], which > fails the build: > here is the list: https://analysis.apache.org/jenkins/view/maven/ > and one faiklure example: > https://analysis.apache.org/jenkins/view/maven/job/maven-parent-pom/113/console > Please update Maven version used to a newer one (3.0.5 or more) > notice: if build failures could be sent to notificati...@maven.apache.org, > please, this would ease detection of such failing analysis builds by Maven > team -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BUILDS-106) update Maven version for Maven builds
[ https://issues.apache.org/jira/browse/BUILDS-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704841#comment-14704841 ] Gavin commented on BUILDS-106: -- asf-paren-pom: maven 3.0.3 -> maven 3.2.2 :: jdk 1.6.0_24 -> no change :: mailto set to notificati...@maven.apache.org ran a build and it passed. > update Maven version for Maven builds > - > > Key: BUILDS-106 > URL: https://issues.apache.org/jira/browse/BUILDS-106 > Project: Infra Build Platform > Issue Type: Task > Components: Analysis >Reporter: Hervé Boutemy > > it seems a good number of jobs for Maven run with Maven [3.0,3.0.3], which > fails the build: > here is the list: https://analysis.apache.org/jenkins/view/maven/ > and one faiklure example: > https://analysis.apache.org/jenkins/view/maven/job/maven-parent-pom/113/console > Please update Maven version used to a newer one (3.0.5 or more) > notice: if build failures could be sent to notificati...@maven.apache.org, > please, this would ease detection of such failing analysis builds by Maven > team -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BUILDS-106) update Maven version for Maven builds
[ https://issues.apache.org/jira/browse/BUILDS-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704842#comment-14704842 ] Gavin commented on BUILDS-106: -- let me know if I need to do anything this side for the failures or if its all up to the project to fix. > update Maven version for Maven builds > - > > Key: BUILDS-106 > URL: https://issues.apache.org/jira/browse/BUILDS-106 > Project: Infra Build Platform > Issue Type: Task > Components: Analysis >Reporter: Hervé Boutemy > > it seems a good number of jobs for Maven run with Maven [3.0,3.0.3], which > fails the build: > here is the list: https://analysis.apache.org/jenkins/view/maven/ > and one faiklure example: > https://analysis.apache.org/jenkins/view/maven/job/maven-parent-pom/113/console > Please update Maven version used to a newer one (3.0.5 or more) > notice: if build failures could be sent to notificati...@maven.apache.org, > please, this would ease detection of such failing analysis builds by Maven > team -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BUILDS-89) JDK 1.7 is not found on Jenkins for Apache Tamaya
[ https://issues.apache.org/jira/browse/BUILDS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin closed BUILDS-89. --- Resolution: Fixed ok so I installed a toolchains.xml into ubuntu3 and tied your build to it. ran fine and passed. https://builds.apache.org/job/Tamaya-Master-branch/ I will roll this out to the rest of the slaves, in the meantime stick to ubuntu3. > JDK 1.7 is not found on Jenkins for Apache Tamaya > - > > Key: BUILDS-89 > URL: https://issues.apache.org/jira/browse/BUILDS-89 > Project: Infra Build Platform > Issue Type: Bug > Components: Jenkins > Environment: https://builds.apache.org/job/Tamaya-Master-branch/ >Reporter: Oliver B. Fischer >Assignee: Gavin >Priority: Critical > > We use JDK 7 and JDK 8 in our build. But since some weeks JDK 7 is not found > on Jenkins. Can someone help us to find out the reason for that? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Name resolution issue on ubuntu jenkins slaves?
So the Yahoo! provided slaves (H*, ubuntu-*) do have Level3's DNS servers in /etc/resolv.conf - we don't actually control that directly, it's part of the system setup Y! uses, I think. I'll ping our contact at Y! about changing them to use more standard DNS. A. On Thu, Aug 20, 2015 at 6:15 AM, Gavin McDonald wrote: > > > On 20 Aug 2015, at 4:03 am, David Nalley wrote: > > > > Whatever we are using for DNS on those build slaves is redirecting any > > NXDOMAIN to a Level3 search domain. > > > > Is this happening on the ubuntu-* slaves or on the dynamic build slaves? > > It was mentioned earlier : > > > I am certainly seeing this on slaves ubuntu4 > > through ubuntu6. > > I have seen passes and failures on the dynamic slaves too, so seems > inconsistent. > > Gav… > > > > > --David > > > > On Mon, Aug 10, 2015 at 8:59 AM, Keith W wrote: > >> Hello Apache builds, > >> > >> We (Apache Qpid) have been seeing a test fail on the Ubuntu Jenkins > slaves > >> since August 9th. The test verifies the behaviour of the code when the > >> user specifies a hostname that does not exist in DNS. For this purpose, > >> the test uses a random name 'hg3sgaaw4lgihjs' (without hierarchal part) > >> which is assumed not resolve. This test is longstanding and has been > >> running on the slaves for many years without issue. > >> > >> Between August 9th and 10th, something appears to have changed on the > >> slaves, which is meaning that the lookup of the name is now returning an > >> IP. This is causing the Java test to fail. I've investigated by > >> introducing shell commands into the job, and can see evidence of the > same > >> problem at the UNIX level. I am certainly seeing this on slaves ubuntu4 > >> through ubuntu6. > >> > >> > >> $ host hg3sgaaw4lgihjs > >> hg3sgaaw4lgihjs has address 198.105.244.11 > >> hg3sgaaw4lgihjs has address 198.105.254.11 > >> Host hg3sgaaw4lgihjs not found: 3(NXDOMAIN) > >> > >> $ host hg3sgaaw4lgihjs2 > >> hg3sgaaw4lgihjs2 has address 198.105.244.11 > >> hg3sgaaw4lgihjs2 has address 198.105.254.11 > >> Host hg3sgaaw4lgihjs2 not found: 3(NXDOMAIN) > >> > >> > >> I considered changing the test to use a RFC-2606 Reserved Top Level DNS > >> Names hg3sgaaw4lgihjs.invalid but I notice that it too is resolving to > >> 198.105.244.11 too. The fact that an .invalid address is resolving > makes > >> me suspect there is an environmental problem at the root cause. > >> > >> host hg3sgaaw4lgihjs.invalid > >> hg3sgaaw4lgihjs.invalid has address 198.105.244.11 > >> hg3sgaaw4lgihjs.invalid has address 198.105.254.11 > >> Host hg3sgaaw4lgihjs.invalid not found: 3(NXDOMAIN) > >> > >> Is there a name resolution issue affecting these hosts? > >> > >> Example job affected by the issue: > >> > >> > https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-Test-JDK1.8/ > >> > >> Kind regards, Keith Wall. > > >
Re: Build slave capacity on builds.apache.org
On Thu, Aug 20, 2015 at 6:32 AM, Gavin McDonald wrote: > >> On 20 Aug 2015, at 2:18 am, David Nalley wrote: >> >> > >> ...how many additional >> build nodes/executors will satiate our current demand for capacity? > > I know this was not aimed at me but I’ll give my opinion. > > From what I’ve seen of the current growth over the last year; and to allow for > future expansion over the next 2 years I would look to get another 20 static > slave machines/vms , each running 2 executors. > Just for some perspective keep in mind that we are 4 months into the current budget year. The total hardware/cloud budget for all of the ASF is 90k. 20 additional physical nodes is a 200k capital outlay if we purchase hardware, and 50k/yr amortized over 4 years, and likely another rack of power and cooling. At RAX that's $4321/month and $51,852/year. The cheapest option would be going to the group Daniel found in France for ELK, while not nearly as flexible as a true cloud system, we could run 20 nodes there for $1500 per month, or $18,000 per year. And that's an increase on what we are currently spending, not the total. We can ask some of our infra sponsors for that, but we need to plan that out, and deal with the risk that involves. We are already heavily dependent on a single sponsor for the vast majority of our physical CI nodes. That said, we really need to understand the demand first before we decide to spend tens or hundreds of thousands of dollars. In many ways, I expected Travis to decrease load on Jenkins, but we continue to expand. It'd be interesting to see graphs of # of jobs over time, how that's increasing, as well as what projects are currently consuming build time. --David
Re: Build slave capacity on builds.apache.org
On Thu, Aug 20, 2015 at 2:55 AM, Daan Hoogland wrote: > cloudstack...! we started making more intensive use of pull-builders. the > old pull-request build job is replaced by a rat and an analysis job. Maybe > others have done so as well but I am sure we (ACS) are a culprit in this. > I thought all of the ACS PR builds were happening on Travis? --David
Re: Build slave capacity on builds.apache.org
Yeah, I don't think we need physical nodes - leasing hosts somewhere would be perfectly fine. If we did go with physical nodes, I'd guess that five would probably be fine for the next year, split into multiple VMs/containers. I'm working on getting stats/graphs - should hopefully have that ready in a week or so. A. On Thu, Aug 20, 2015 at 11:01 AM, David Nalley wrote: > On Thu, Aug 20, 2015 at 6:32 AM, Gavin McDonald > wrote: > > > >> On 20 Aug 2015, at 2:18 am, David Nalley wrote: > >> > >> > > > >> ...how many additional > >> build nodes/executors will satiate our current demand for capacity? > > > > I know this was not aimed at me but I’ll give my opinion. > > > > From what I’ve seen of the current growth over the last year; and to > allow for > > future expansion over the next 2 years I would look to get another 20 > static > > slave machines/vms , each running 2 executors. > > > > Just for some perspective keep in mind that we are 4 months into the > current budget year. The total hardware/cloud budget for all of the > ASF is 90k. > 20 additional physical nodes is a 200k capital outlay if we purchase > hardware, and 50k/yr amortized over 4 years, and likely another rack > of power and cooling. > At RAX that's $4321/month and $51,852/year. > The cheapest option would be going to the group Daniel found in France > for ELK, while not nearly as flexible as a true cloud system, we could > run 20 nodes there for $1500 per month, or $18,000 per year. > And that's an increase on what we are currently spending, not the > total. We can ask some of our infra sponsors for that, but we need to > plan that out, and deal with the risk that involves. We are already > heavily dependent on a single sponsor for the vast majority of our > physical CI nodes. > > That said, we really need to understand the demand first before we > decide to spend tens or hundreds of thousands of dollars. In many > ways, I expected Travis to decrease load on Jenkins, but we continue > to expand. It'd be interesting to see graphs of # of jobs over time, > how that's increasing, as well as what projects are currently > consuming build time. > > --David >
Re: Build slave capacity on builds.apache.org
On Thu, Aug 20, 2015 at 5:02 PM, David Nalley wrote: > On Thu, Aug 20, 2015 at 2:55 AM, Daan Hoogland > wrote: > > cloudstack...! > > I thought all of the ACS PR builds were happening on Travis? > only smoke tests, no code analysis . > --David > -- Daan
[jira] [Created] (BUILDS-110) Set up Jenkins job linter/rules enforcer
Andrew Bayer created BUILDS-110: --- Summary: Set up Jenkins job linter/rules enforcer Key: BUILDS-110 URL: https://issues.apache.org/jira/browse/BUILDS-110 Project: Infra Build Platform Issue Type: Task Reporter: Andrew Bayer Assignee: Andrew Bayer -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BUILDS-111) Auto-cleanup job for orphaned Java processes on builds.a.o
Andrew Bayer created BUILDS-111: --- Summary: Auto-cleanup job for orphaned Java processes on builds.a.o Key: BUILDS-111 URL: https://issues.apache.org/jira/browse/BUILDS-111 Project: Infra Build Platform Issue Type: Task Reporter: Andrew Bayer Assignee: Andrew Bayer -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BUILDS-85) Could not generate DH keypair / peer not authenticated
[ https://issues.apache.org/jira/browse/BUILDS-85?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14705228#comment-14705228 ] Gavin commented on BUILDS-85: - [~mgrigorov] and everyone else here - I've deployed a toolchains.xml across all nodes - specifies java 1.5, 1.6, 1.7 and 1.8 (oracle) so you should be all set. > Could not generate DH keypair / peer not authenticated > --- > > Key: BUILDS-85 > URL: https://issues.apache.org/jira/browse/BUILDS-85 > Project: Infra Build Platform > Issue Type: Bug > Components: Jenkins >Reporter: Andreas Lehmkühler >Assignee: Tony Stevenson > Attachments: SSLPoke.java > > > We're getting this since june 10th: > [INFO] --- maven-deploy-plugin:2.6:deploy (default-deploy) @ pdfbox-parent --- > Downloading:https://repository.apache.org/content/repositories/snapshots/org/apache/pdfbox/pdfbox-parent/1.8.10-SNAPSHOT/maven-metadata.xml > [WARNING] Could not transfer metadata > org.apache.pdfbox:pdfbox-parent:1.8.10-SNAPSHOT/maven-metadata.xml from/to > apache.snapshots.https > (https://repository.apache.org/content/repositories/snapshots): Error > transferring file: java.lang.RuntimeException: Could not generate DH keypair > and this: > [INFO] --- maven-deploy-plugin:2.8.2:deploy (default-deploy) @ pdfbox-parent > --- > Downloading:https://repository.apache.org/content/repositories/snapshots/org/apache/pdfbox/pdfbox-parent/2.0.0-SNAPSHOT/maven-metadata.xml > [WARNING] Could not transfer metadata > org.apache.pdfbox:pdfbox-parent:2.0.0-SNAPSHOT/maven-metadata.xml from/to > apache.snapshots.https > (https://repository.apache.org/content/repositories/snapshots): peer not > authenticated > The issue seems to be jdk related as only those builds using java 1.6.0_37 > (unlimited security) are failing. I've reconfigured the trunk build to use > java 7 and everything works fine, as well as our jdk7 based branch build. > Any ideas? Maybe a plugin update which doesn't work with java6? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BUILDS-111) Auto-cleanup job for orphaned Java processes on builds.a.o
[ https://issues.apache.org/jira/browse/BUILDS-111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14705269#comment-14705269 ] Andrew Bayer commented on BUILDS-111: - Gonna need to restart Jenkins to install the groovy plugin to do some magic to find the build ID for any jobs running on the slave to make sure this doesn't step on other jobs running on the slave at the same time... > Auto-cleanup job for orphaned Java processes on builds.a.o > -- > > Key: BUILDS-111 > URL: https://issues.apache.org/jira/browse/BUILDS-111 > Project: Infra Build Platform > Issue Type: Task >Reporter: Andrew Bayer >Assignee: Andrew Bayer > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BUILDS-108) Problem with ServiceMix-Bundles Jenkins job
[ https://issues.apache.org/jira/browse/BUILDS-108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14705313#comment-14705313 ] Krzysztof Sobkowiak commented on BUILDS-108: It seems to work correctly now > Problem with ServiceMix-Bundles Jenkins job > --- > > Key: BUILDS-108 > URL: https://issues.apache.org/jira/browse/BUILDS-108 > Project: Infra Build Platform > Issue Type: Bug >Reporter: Krzysztof Sobkowiak > > Looking at the last job runs in > https://builds.apache.org/view/All/job/ServiceMix-Bundles I can see some > modules still counting since 18 hours, but the whole build seems to be > aborted. Could you please check the build and kill the modules if necessary? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (BUILDS-108) Problem with ServiceMix-Bundles Jenkins job
[ https://issues.apache.org/jira/browse/BUILDS-108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Bayer reassigned BUILDS-108: --- Assignee: Andrew Bayer > Problem with ServiceMix-Bundles Jenkins job > --- > > Key: BUILDS-108 > URL: https://issues.apache.org/jira/browse/BUILDS-108 > Project: Infra Build Platform > Issue Type: Bug >Reporter: Krzysztof Sobkowiak >Assignee: Andrew Bayer > > Looking at the last job runs in > https://builds.apache.org/view/All/job/ServiceMix-Bundles I can see some > modules still counting since 18 hours, but the whole build seems to be > aborted. Could you please check the build and kill the modules if necessary? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Name resolution issue on ubuntu jenkins slaves?
Yup, http://drewgraybeal.blogspot.com/2015/05/level-3-dns-hijacking-4222-and-others.html - level3 is hijacking NXDOMAIN now. I think Y!'s DHCP is setting those DNS servers? Rajiv, can you comment? A. On Thu, Aug 20, 2015 at 10:32 AM, Andrew Bayer wrote: > So the Yahoo! provided slaves (H*, ubuntu-*) do have Level3's DNS servers > in /etc/resolv.conf - we don't actually control that directly, it's part of > the system setup Y! uses, I think. I'll ping our contact at Y! about > changing them to use more standard DNS. > > A. > > On Thu, Aug 20, 2015 at 6:15 AM, Gavin McDonald > wrote: > >> >> > On 20 Aug 2015, at 4:03 am, David Nalley wrote: >> > >> > Whatever we are using for DNS on those build slaves is redirecting any >> > NXDOMAIN to a Level3 search domain. >> > >> > Is this happening on the ubuntu-* slaves or on the dynamic build slaves? >> >> It was mentioned earlier : >> >> > I am certainly seeing this on slaves ubuntu4 >> > through ubuntu6. >> >> I have seen passes and failures on the dynamic slaves too, so seems >> inconsistent. >> >> Gav… >> >> > >> > --David >> > >> > On Mon, Aug 10, 2015 at 8:59 AM, Keith W wrote: >> >> Hello Apache builds, >> >> >> >> We (Apache Qpid) have been seeing a test fail on the Ubuntu Jenkins >> slaves >> >> since August 9th. The test verifies the behaviour of the code when the >> >> user specifies a hostname that does not exist in DNS. For this >> purpose, >> >> the test uses a random name 'hg3sgaaw4lgihjs' (without hierarchal part) >> >> which is assumed not resolve. This test is longstanding and has been >> >> running on the slaves for many years without issue. >> >> >> >> Between August 9th and 10th, something appears to have changed on the >> >> slaves, which is meaning that the lookup of the name is now returning >> an >> >> IP. This is causing the Java test to fail. I've investigated by >> >> introducing shell commands into the job, and can see evidence of the >> same >> >> problem at the UNIX level. I am certainly seeing this on slaves >> ubuntu4 >> >> through ubuntu6. >> >> >> >> >> >> $ host hg3sgaaw4lgihjs >> >> hg3sgaaw4lgihjs has address 198.105.244.11 >> >> hg3sgaaw4lgihjs has address 198.105.254.11 >> >> Host hg3sgaaw4lgihjs not found: 3(NXDOMAIN) >> >> >> >> $ host hg3sgaaw4lgihjs2 >> >> hg3sgaaw4lgihjs2 has address 198.105.244.11 >> >> hg3sgaaw4lgihjs2 has address 198.105.254.11 >> >> Host hg3sgaaw4lgihjs2 not found: 3(NXDOMAIN) >> >> >> >> >> >> I considered changing the test to use a RFC-2606 Reserved Top Level DNS >> >> Names hg3sgaaw4lgihjs.invalid but I notice that it too is resolving to >> >> 198.105.244.11 too. The fact that an .invalid address is resolving >> makes >> >> me suspect there is an environmental problem at the root cause. >> >> >> >> host hg3sgaaw4lgihjs.invalid >> >> hg3sgaaw4lgihjs.invalid has address 198.105.244.11 >> >> hg3sgaaw4lgihjs.invalid has address 198.105.254.11 >> >> Host hg3sgaaw4lgihjs.invalid not found: 3(NXDOMAIN) >> >> >> >> Is there a name resolution issue affecting these hosts? >> >> >> >> Example job affected by the issue: >> >> >> >> >> https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-Test-JDK1.8/ >> >> >> >> Kind regards, Keith Wall. >> >> >> >
[jira] [Commented] (BUILDS-108) Problem with ServiceMix-Bundles Jenkins job
[ https://issues.apache.org/jira/browse/BUILDS-108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14705326#comment-14705326 ] Andrew Bayer commented on BUILDS-108: - A whole lot of things were in messed up states - hung builds, overloaded slaves, etc. I've restarted Jenkins and cleaned out orphaned java processes on the H* and ubuntu-* slaves, so hopefully things will be cleaner now. > Problem with ServiceMix-Bundles Jenkins job > --- > > Key: BUILDS-108 > URL: https://issues.apache.org/jira/browse/BUILDS-108 > Project: Infra Build Platform > Issue Type: Bug >Reporter: Krzysztof Sobkowiak >Assignee: Andrew Bayer > > Looking at the last job runs in > https://builds.apache.org/view/All/job/ServiceMix-Bundles I can see some > modules still counting since 18 hours, but the whole build seems to be > aborted. Could you please check the build and kill the modules if necessary? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Name resolution issue on ubuntu jenkins slaves?
FYI, I've manually overwritten /etc/resolv.conf on all the H* and ubuntu-* slaves with something more valid. A. On Thu, Aug 20, 2015 at 1:16 PM, Andrew Bayer wrote: > Yup, > http://drewgraybeal.blogspot.com/2015/05/level-3-dns-hijacking-4222-and-others.html > - > level3 is hijacking NXDOMAIN now. I think Y!'s DHCP is setting those DNS > servers? Rajiv, can you comment? > > A. > > On Thu, Aug 20, 2015 at 10:32 AM, Andrew Bayer > wrote: > >> So the Yahoo! provided slaves (H*, ubuntu-*) do have Level3's DNS servers >> in /etc/resolv.conf - we don't actually control that directly, it's part of >> the system setup Y! uses, I think. I'll ping our contact at Y! about >> changing them to use more standard DNS. >> >> A. >> >> On Thu, Aug 20, 2015 at 6:15 AM, Gavin McDonald >> wrote: >> >>> >>> > On 20 Aug 2015, at 4:03 am, David Nalley wrote: >>> > >>> > Whatever we are using for DNS on those build slaves is redirecting any >>> > NXDOMAIN to a Level3 search domain. >>> > >>> > Is this happening on the ubuntu-* slaves or on the dynamic build >>> slaves? >>> >>> It was mentioned earlier : >>> >>> > I am certainly seeing this on slaves ubuntu4 >>> > through ubuntu6. >>> >>> I have seen passes and failures on the dynamic slaves too, so seems >>> inconsistent. >>> >>> Gav… >>> >>> > >>> > --David >>> > >>> > On Mon, Aug 10, 2015 at 8:59 AM, Keith W wrote: >>> >> Hello Apache builds, >>> >> >>> >> We (Apache Qpid) have been seeing a test fail on the Ubuntu Jenkins >>> slaves >>> >> since August 9th. The test verifies the behaviour of the code when >>> the >>> >> user specifies a hostname that does not exist in DNS. For this >>> purpose, >>> >> the test uses a random name 'hg3sgaaw4lgihjs' (without hierarchal >>> part) >>> >> which is assumed not resolve. This test is longstanding and has been >>> >> running on the slaves for many years without issue. >>> >> >>> >> Between August 9th and 10th, something appears to have changed on the >>> >> slaves, which is meaning that the lookup of the name is now returning >>> an >>> >> IP. This is causing the Java test to fail. I've investigated by >>> >> introducing shell commands into the job, and can see evidence of the >>> same >>> >> problem at the UNIX level. I am certainly seeing this on slaves >>> ubuntu4 >>> >> through ubuntu6. >>> >> >>> >> >>> >> $ host hg3sgaaw4lgihjs >>> >> hg3sgaaw4lgihjs has address 198.105.244.11 >>> >> hg3sgaaw4lgihjs has address 198.105.254.11 >>> >> Host hg3sgaaw4lgihjs not found: 3(NXDOMAIN) >>> >> >>> >> $ host hg3sgaaw4lgihjs2 >>> >> hg3sgaaw4lgihjs2 has address 198.105.244.11 >>> >> hg3sgaaw4lgihjs2 has address 198.105.254.11 >>> >> Host hg3sgaaw4lgihjs2 not found: 3(NXDOMAIN) >>> >> >>> >> >>> >> I considered changing the test to use a RFC-2606 Reserved Top Level >>> DNS >>> >> Names hg3sgaaw4lgihjs.invalid but I notice that it too is resolving >>> to >>> >> 198.105.244.11 too. The fact that an .invalid address is resolving >>> makes >>> >> me suspect there is an environmental problem at the root cause. >>> >> >>> >> host hg3sgaaw4lgihjs.invalid >>> >> hg3sgaaw4lgihjs.invalid has address 198.105.244.11 >>> >> hg3sgaaw4lgihjs.invalid has address 198.105.254.11 >>> >> Host hg3sgaaw4lgihjs.invalid not found: 3(NXDOMAIN) >>> >> >>> >> Is there a name resolution issue affecting these hosts? >>> >> >>> >> Example job affected by the issue: >>> >> >>> >> >>> https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-Test-JDK1.8/ >>> >> >>> >> Kind regards, Keith Wall. >>> >>> >>> >> >
[jira] [Commented] (BUILDS-108) Problem with ServiceMix-Bundles Jenkins job
[ https://issues.apache.org/jira/browse/BUILDS-108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14705351#comment-14705351 ] Krzysztof Sobkowiak commented on BUILDS-108: [~abayer] thanks for your help > Problem with ServiceMix-Bundles Jenkins job > --- > > Key: BUILDS-108 > URL: https://issues.apache.org/jira/browse/BUILDS-108 > Project: Infra Build Platform > Issue Type: Bug >Reporter: Krzysztof Sobkowiak >Assignee: Andrew Bayer > > Looking at the last job runs in > https://builds.apache.org/view/All/job/ServiceMix-Bundles I can see some > modules still counting since 18 hours, but the whole build seems to be > aborted. Could you please check the build and kill the modules if necessary? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
FYI - disabled timer triggers across the board on builds.apache.org
I noticed a lot of jobs that were running every day regardless of whether anything changed and eating up a lot of executors, so I bulk-removed all of them, changing those jobs to poll for changes hourly instead. If your project has one or two jobs that you need to run daily whether there are code changes or not, you can re-enable the timer, but please do not do that for more than a couple jobs, and please do not do it for jobs that take longer than half an hour. A.
[jira] [Commented] (BUILDS-85) Could not generate DH keypair / peer not authenticated
[ https://issues.apache.org/jira/browse/BUILDS-85?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14705384#comment-14705384 ] Martin Grigorov commented on BUILDS-85: --- [~ipv6guru] You rock, man! > Could not generate DH keypair / peer not authenticated > --- > > Key: BUILDS-85 > URL: https://issues.apache.org/jira/browse/BUILDS-85 > Project: Infra Build Platform > Issue Type: Bug > Components: Jenkins >Reporter: Andreas Lehmkühler >Assignee: Tony Stevenson > Attachments: SSLPoke.java > > > We're getting this since june 10th: > [INFO] --- maven-deploy-plugin:2.6:deploy (default-deploy) @ pdfbox-parent --- > Downloading:https://repository.apache.org/content/repositories/snapshots/org/apache/pdfbox/pdfbox-parent/1.8.10-SNAPSHOT/maven-metadata.xml > [WARNING] Could not transfer metadata > org.apache.pdfbox:pdfbox-parent:1.8.10-SNAPSHOT/maven-metadata.xml from/to > apache.snapshots.https > (https://repository.apache.org/content/repositories/snapshots): Error > transferring file: java.lang.RuntimeException: Could not generate DH keypair > and this: > [INFO] --- maven-deploy-plugin:2.8.2:deploy (default-deploy) @ pdfbox-parent > --- > Downloading:https://repository.apache.org/content/repositories/snapshots/org/apache/pdfbox/pdfbox-parent/2.0.0-SNAPSHOT/maven-metadata.xml > [WARNING] Could not transfer metadata > org.apache.pdfbox:pdfbox-parent:2.0.0-SNAPSHOT/maven-metadata.xml from/to > apache.snapshots.https > (https://repository.apache.org/content/repositories/snapshots): peer not > authenticated > The issue seems to be jdk related as only those builds using java 1.6.0_37 > (unlimited security) are failing. I've reconfigured the trunk build to use > java 7 and everything works fine, as well as our jdk7 based branch build. > Any ideas? Maybe a plugin update which doesn't work with java6? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BUILDS-106) update Maven version for Maven builds
[ https://issues.apache.org/jira/browse/BUILDS-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14705410#comment-14705410 ] Hervé Boutemy commented on BUILDS-106: -- thank you Gavin here are next steps, please: 1. for every job, paticularly those that fail on rat, please add "-Drat.skip=true" (for such analysis jobs, no need to bother with rat that sometime fails on screwed workspace) 2. tomcat-maven-plugin doesn't belong to Maven PMC but Tomcat PMC: please remove the mail notification (or send to Tomcat list if they have an equivalent list?) 3. for jobs that fail because of missing pom.xml, there is something strange at Jenkins checkout, since the file is in svn but isn't checked out: I don't know what's wrong in job config for example: failing on analysis Jenkins: https://analysis.apache.org/jenkins/view/maven/job/maven-plugin-tools/90/console but the same on builds jenkins: https://builds.apache.org/view/M-R/view/Maven/job/maven-plugin-tools/244/consoleFull here, from the same svn url, pom.xml is checked-out I still need to investigate how some jobs have compilation errors while builds corresponding jobs don't have such compilation failures: but analysis jenkins seems a little strage (in addition to being somewhat black box) > update Maven version for Maven builds > - > > Key: BUILDS-106 > URL: https://issues.apache.org/jira/browse/BUILDS-106 > Project: Infra Build Platform > Issue Type: Task > Components: Analysis >Reporter: Hervé Boutemy > > it seems a good number of jobs for Maven run with Maven [3.0,3.0.3], which > fails the build: > here is the list: https://analysis.apache.org/jenkins/view/maven/ > and one faiklure example: > https://analysis.apache.org/jenkins/view/maven/job/maven-parent-pom/113/console > Please update Maven version used to a newer one (3.0.5 or more) > notice: if build failures could be sent to notificati...@maven.apache.org, > please, this would ease detection of such failing analysis builds by Maven > team -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Build slave capacity on builds.apache.org
So it looks like part of the problem is that HBase builds have been hanging/causing slaves to barf out due to orphaned java processes eating up resources. They're looking into this over at https://issues.apache.org/jira/browse/INFRA-10150. On Thu, Aug 20, 2015 at 11:44 AM, Daan Hoogland wrote: > > > On Thu, Aug 20, 2015 at 5:02 PM, David Nalley wrote: > >> On Thu, Aug 20, 2015 at 2:55 AM, Daan Hoogland >> wrote: >> > cloudstack...! >> >> I thought all of the ACS PR builds were happening on Travis? >> > only smoke tests, no code analysis > . > > >> --David >> > > > > -- > Daan >
[jira] [Assigned] (BUILDS-14) Install RVM on ubuntu-* slaves
[ https://issues.apache.org/jira/browse/BUILDS-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin reassigned BUILDS-14: --- Assignee: Gavin > Install RVM on ubuntu-* slaves > -- > > Key: BUILDS-14 > URL: https://issues.apache.org/jira/browse/BUILDS-14 > Project: Infra Build Platform > Issue Type: Task > Components: Jenkins >Reporter: Tammo van Lessen >Assignee: Gavin > > Hi, > we're using Apache Buildr for the ODE builds and it needs Ruby/JRuby to run. > RVM helps to install and maintain the installed rubies and lets project > choose the version they need. It is already installed on the ubuntu slaves > without the dash. > Install instructions can be found at https://rvm.io/ -- the best way is to > run the installation as the jenkins user as it is then installed into the > user's home directory and no sudo foo is required. > Thanks! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BUILDS-14) Install RVM on ubuntu-* slaves
[ https://issues.apache.org/jira/browse/BUILDS-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin closed BUILDS-14. --- Resolution: Fixed yeah done a while ago. > Install RVM on ubuntu-* slaves > -- > > Key: BUILDS-14 > URL: https://issues.apache.org/jira/browse/BUILDS-14 > Project: Infra Build Platform > Issue Type: Task > Components: Jenkins >Reporter: Tammo van Lessen >Assignee: Gavin > > Hi, > we're using Apache Buildr for the ODE builds and it needs Ruby/JRuby to run. > RVM helps to install and maintain the installed rubies and lets project > choose the version they need. It is already installed on the ubuntu slaves > without the dash. > Install instructions can be found at https://rvm.io/ -- the best way is to > run the installation as the jenkins user as it is then installed into the > user's home directory and no sudo foo is required. > Thanks! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BUILDS-85) Could not generate DH keypair / peer not authenticated
[
https://issues.apache.org/jira/browse/BUILDS-85?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14705777#comment-14705777
]
Martin Grigorov commented on BUILDS-85:
---
[~ipv6guru] Just tried it and the build at BuildBot (slave18) fails with:
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-toolchains-plugin:1.1:toolchain (default) on
project wicket-parent: Cannot find matching toolchain definitions for the
following toolchain types:
[ERROR] jdk [ vendor='oracle' version='1.6' ]
Apparently the vendor/version pair is not oracle/1.6.
My local ~/.m2/toolchains.xml looks like:
{code}
jdk
1.6
oracle
/home/martin/devel/java-6/
{code}
Please check how your looks.
My change to Wicket's pom.xml is at
https://git1-us-west.apache.org/repos/asf?p=wicket.git;a=commitdiff;h=5b6397ba;hp=62f95feb6226dfe9bdad37a8760250405e560654
The full logs from BuildBot could be found at
https://ci.apache.org/builders/wicket-branch-6.x/builds/30/steps/compile/logs/stdio
> Could not generate DH keypair / peer not authenticated
> ---
>
> Key: BUILDS-85
> URL: https://issues.apache.org/jira/browse/BUILDS-85
> Project: Infra Build Platform
> Issue Type: Bug
> Components: Jenkins
>Reporter: Andreas Lehmkühler
>Assignee: Tony Stevenson
> Attachments: SSLPoke.java
>
>
> We're getting this since june 10th:
> [INFO] --- maven-deploy-plugin:2.6:deploy (default-deploy) @ pdfbox-parent ---
> Downloading:https://repository.apache.org/content/repositories/snapshots/org/apache/pdfbox/pdfbox-parent/1.8.10-SNAPSHOT/maven-metadata.xml
> [WARNING] Could not transfer metadata
> org.apache.pdfbox:pdfbox-parent:1.8.10-SNAPSHOT/maven-metadata.xml from/to
> apache.snapshots.https
> (https://repository.apache.org/content/repositories/snapshots): Error
> transferring file: java.lang.RuntimeException: Could not generate DH keypair
> and this:
> [INFO] --- maven-deploy-plugin:2.8.2:deploy (default-deploy) @ pdfbox-parent
> ---
> Downloading:https://repository.apache.org/content/repositories/snapshots/org/apache/pdfbox/pdfbox-parent/2.0.0-SNAPSHOT/maven-metadata.xml
> [WARNING] Could not transfer metadata
> org.apache.pdfbox:pdfbox-parent:2.0.0-SNAPSHOT/maven-metadata.xml from/to
> apache.snapshots.https
> (https://repository.apache.org/content/repositories/snapshots): peer not
> authenticated
> The issue seems to be jdk related as only those builds using java 1.6.0_37
> (unlimited security) are failing. I've reconfigured the trunk build to use
> java 7 and everything works fine, as well as our jdk7 based branch build.
> Any ideas? Maybe a plugin update which doesn't work with java6?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BUILDS-85) Could not generate DH keypair / peer not authenticated
[ https://issues.apache.org/jira/browse/BUILDS-85?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14705806#comment-14705806 ] Gavin commented on BUILDS-85: - check it out for yourself: https://github.com/apache/infrastructure-puppet/blob/deployment/modules/build_slaves/files/toolchains.xml > Could not generate DH keypair / peer not authenticated > --- > > Key: BUILDS-85 > URL: https://issues.apache.org/jira/browse/BUILDS-85 > Project: Infra Build Platform > Issue Type: Bug > Components: Jenkins >Reporter: Andreas Lehmkühler >Assignee: Tony Stevenson > Attachments: SSLPoke.java > > > We're getting this since june 10th: > [INFO] --- maven-deploy-plugin:2.6:deploy (default-deploy) @ pdfbox-parent --- > Downloading:https://repository.apache.org/content/repositories/snapshots/org/apache/pdfbox/pdfbox-parent/1.8.10-SNAPSHOT/maven-metadata.xml > [WARNING] Could not transfer metadata > org.apache.pdfbox:pdfbox-parent:1.8.10-SNAPSHOT/maven-metadata.xml from/to > apache.snapshots.https > (https://repository.apache.org/content/repositories/snapshots): Error > transferring file: java.lang.RuntimeException: Could not generate DH keypair > and this: > [INFO] --- maven-deploy-plugin:2.8.2:deploy (default-deploy) @ pdfbox-parent > --- > Downloading:https://repository.apache.org/content/repositories/snapshots/org/apache/pdfbox/pdfbox-parent/2.0.0-SNAPSHOT/maven-metadata.xml > [WARNING] Could not transfer metadata > org.apache.pdfbox:pdfbox-parent:2.0.0-SNAPSHOT/maven-metadata.xml from/to > apache.snapshots.https > (https://repository.apache.org/content/repositories/snapshots): peer not > authenticated > The issue seems to be jdk related as only those builds using java 1.6.0_37 > (unlimited security) are failing. I've reconfigured the trunk build to use > java 7 and everything works fine, as well as our jdk7 based branch build. > Any ideas? Maybe a plugin update which doesn't work with java6? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BUILDS-85) Could not generate DH keypair / peer not authenticated
[ https://issues.apache.org/jira/browse/BUILDS-85?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14705811#comment-14705811 ] Gavin commented on BUILDS-85: - Oh I see whats happening. Most people in this ticket have been talking about Jenkins, so I've deployed toolchain to jenkins. Do me a favour please and open a new ticket and I'll do the same for Buildbot ;) > Could not generate DH keypair / peer not authenticated > --- > > Key: BUILDS-85 > URL: https://issues.apache.org/jira/browse/BUILDS-85 > Project: Infra Build Platform > Issue Type: Bug > Components: Jenkins >Reporter: Andreas Lehmkühler >Assignee: Tony Stevenson > Attachments: SSLPoke.java > > > We're getting this since june 10th: > [INFO] --- maven-deploy-plugin:2.6:deploy (default-deploy) @ pdfbox-parent --- > Downloading:https://repository.apache.org/content/repositories/snapshots/org/apache/pdfbox/pdfbox-parent/1.8.10-SNAPSHOT/maven-metadata.xml > [WARNING] Could not transfer metadata > org.apache.pdfbox:pdfbox-parent:1.8.10-SNAPSHOT/maven-metadata.xml from/to > apache.snapshots.https > (https://repository.apache.org/content/repositories/snapshots): Error > transferring file: java.lang.RuntimeException: Could not generate DH keypair > and this: > [INFO] --- maven-deploy-plugin:2.8.2:deploy (default-deploy) @ pdfbox-parent > --- > Downloading:https://repository.apache.org/content/repositories/snapshots/org/apache/pdfbox/pdfbox-parent/2.0.0-SNAPSHOT/maven-metadata.xml > [WARNING] Could not transfer metadata > org.apache.pdfbox:pdfbox-parent:2.0.0-SNAPSHOT/maven-metadata.xml from/to > apache.snapshots.https > (https://repository.apache.org/content/repositories/snapshots): peer not > authenticated > The issue seems to be jdk related as only those builds using java 1.6.0_37 > (unlimited security) are failing. I've reconfigured the trunk build to use > java 7 and everything works fine, as well as our jdk7 based branch build. > Any ideas? Maybe a plugin update which doesn't work with java6? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
RE: FYI - disabled timer triggers across the board on builds.apache.org
Hi Andrew, see my mail thread about the Lucene builds this morning with Gav: > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Thursday, August 20, 2015 11:28 AM > To: builds@apache.org > Subject: Re: Number of build prozessors on Lucene node > > Hi, > > The backlog of Lucene is generally large. That's wanted, because we don't > trigger builds based on commits; just to always have one build of everything > in the queue we do it time-based. Because of our pseudorandomized test > infrastructure, the build slave should never be out of work because every > run may trigger a bug in Lucene or the Java VM. Because Jenkins never > triggers the same job multiple times, it's perfectly fine to have at least one > job of all in the queue. > > If you don't want your statistics broken, you may exclude the lucene slave > from counting towards queue size. It's completely on its own and only allows > ecplicitely assigned jobs. > > You can always remove triggered builds from queu (I sometimes do this for > testing puposes or to trigger a special build manually). They get queued > anyways a while later if deleted. > > Uwe > > Am 20. August 2015 11:15:17 MESZ, schrieb Gavin McDonald > : > > > >> On 20 Aug 2015, at 9:17 am, Uwe Schindler > >wrote: > >> > >> Hi, > >> > >> could someone please change back the number of "build processors" for > >the "lucene" Jenkins node to 1? Currently it executes always 2 Jobs in > >parallel. The underlying server only has 4 cpu cores and the Lucene Job > >configuration is done in a way that it uses all available CPU nodes, so > >running 2 builds in parallel on is in most cases not a good idea. There > >are in fact some jobs, that don't require much CPU or are not > >multithreaded (like artifact builds), but those are generally quick. > >The main tasks taking several hours to execute are very CPU intensive - > >the reason why we have an own slave. > >> > >> Any information why this was changed? Unfortunately I don't have the > >power to configure this correctly. > > > >I changed it short term to help with backlog. > >When I did this (2 days ago) there were 166 builds in the queue, over > >30 of those were waiting on the Lucene node. > > > >Will change it back now. > > > >HTH > > > >Gav… > > > >> > >> Uwe > >> As said before, the Lucene builds are only affecting the Lucene slave and this one should be used as much as possible, I reverted to the timer triggers. I am glad that you only changed builds which used timers, otherwise the whole cross-project dependency tree in Lucene would have been broken. So it was only a few jobs to start timely (per commit did not make sense for "nightly" jobs). Uwe - Uwe Schindler uschind...@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ > -Original Message- > From: Andrew Bayer [mailto:andrew.ba...@gmail.com] > Sent: Thursday, August 20, 2015 7:33 PM > To: builds@apache.org > Subject: FYI - disabled timer triggers across the board on builds.apache.org > > I noticed a lot of jobs that were running every day regardless of whether > anything changed and eating up a lot of executors, so I bulk-removed all of > them, changing those jobs to poll for changes hourly instead. If your project > has one or two jobs that you need to run daily whether there are code > changes or not, you can re-enable the timer, but please do not do that for > more than a couple jobs, and please do not do it for jobs that take longer > than half an hour. > > A.
[jira] [Commented] (BUILDS-85) Could not generate DH keypair / peer not authenticated
[ https://issues.apache.org/jira/browse/BUILDS-85?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14705829#comment-14705829 ] Gavin commented on BUILDS-85: - Just added toolchains.xml to buildbot slave 18 so you can try that build again on there. > Could not generate DH keypair / peer not authenticated > --- > > Key: BUILDS-85 > URL: https://issues.apache.org/jira/browse/BUILDS-85 > Project: Infra Build Platform > Issue Type: Bug > Components: Jenkins >Reporter: Andreas Lehmkühler >Assignee: Tony Stevenson > Attachments: SSLPoke.java > > > We're getting this since june 10th: > [INFO] --- maven-deploy-plugin:2.6:deploy (default-deploy) @ pdfbox-parent --- > Downloading:https://repository.apache.org/content/repositories/snapshots/org/apache/pdfbox/pdfbox-parent/1.8.10-SNAPSHOT/maven-metadata.xml > [WARNING] Could not transfer metadata > org.apache.pdfbox:pdfbox-parent:1.8.10-SNAPSHOT/maven-metadata.xml from/to > apache.snapshots.https > (https://repository.apache.org/content/repositories/snapshots): Error > transferring file: java.lang.RuntimeException: Could not generate DH keypair > and this: > [INFO] --- maven-deploy-plugin:2.8.2:deploy (default-deploy) @ pdfbox-parent > --- > Downloading:https://repository.apache.org/content/repositories/snapshots/org/apache/pdfbox/pdfbox-parent/2.0.0-SNAPSHOT/maven-metadata.xml > [WARNING] Could not transfer metadata > org.apache.pdfbox:pdfbox-parent:2.0.0-SNAPSHOT/maven-metadata.xml from/to > apache.snapshots.https > (https://repository.apache.org/content/repositories/snapshots): peer not > authenticated > The issue seems to be jdk related as only those builds using java 1.6.0_37 > (unlimited security) are failing. I've reconfigured the trunk build to use > java 7 and everything works fine, as well as our jdk7 based branch build. > Any ideas? Maybe a plugin update which doesn't work with java6? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
RE: FYI - disabled timer triggers across the board on builds.apache.org
Yeah, that's fine - the only viable way to do this was across the board, so I'm sorry for messing with your jobs. =) On Aug 20, 2015 17:41, "Uwe Schindler" wrote: > Hi Andrew, > > see my mail thread about the Lucene builds this morning with Gav: > > > From: Uwe Schindler [mailto:u...@thetaphi.de] > > Sent: Thursday, August 20, 2015 11:28 AM > > To: builds@apache.org > > Subject: Re: Number of build prozessors on Lucene node > > > > Hi, > > > > The backlog of Lucene is generally large. That's wanted, because we don't > > trigger builds based on commits; just to always have one build of > everything > > in the queue we do it time-based. Because of our pseudorandomized test > > infrastructure, the build slave should never be out of work because every > > run may trigger a bug in Lucene or the Java VM. Because Jenkins never > > triggers the same job multiple times, it's perfectly fine to have at > least one > > job of all in the queue. > > > > If you don't want your statistics broken, you may exclude the lucene > slave > > from counting towards queue size. It's completely on its own and only > allows > > ecplicitely assigned jobs. > > > > You can always remove triggered builds from queu (I sometimes do this for > > testing puposes or to trigger a special build manually). They get queued > > anyways a while later if deleted. > > > > Uwe > > > > Am 20. August 2015 11:15:17 MESZ, schrieb Gavin McDonald > > : > > > > > >> On 20 Aug 2015, at 9:17 am, Uwe Schindler > > >wrote: > > >> > > >> Hi, > > >> > > >> could someone please change back the number of "build processors" for > > >the "lucene" Jenkins node to 1? Currently it executes always 2 Jobs in > > >parallel. The underlying server only has 4 cpu cores and the Lucene Job > > >configuration is done in a way that it uses all available CPU nodes, so > > >running 2 builds in parallel on is in most cases not a good idea. There > > >are in fact some jobs, that don't require much CPU or are not > > >multithreaded (like artifact builds), but those are generally quick. > > >The main tasks taking several hours to execute are very CPU intensive - > > >the reason why we have an own slave. > > >> > > >> Any information why this was changed? Unfortunately I don't have the > > >power to configure this correctly. > > > > > >I changed it short term to help with backlog. > > >When I did this (2 days ago) there were 166 builds in the queue, over > > >30 of those were waiting on the Lucene node. > > > > > >Will change it back now. > > > > > >HTH > > > > > >Gav… > > > > > >> > > >> Uwe > > >> > > As said before, the Lucene builds are only affecting the Lucene slave and > this one should be used as much as possible, I reverted to the timer > triggers. I am glad that you only changed builds which used timers, > otherwise the whole cross-project dependency tree in Lucene would have been > broken. So it was only a few jobs to start timely (per commit did not make > sense for "nightly" jobs). > > Uwe > > - > Uwe Schindler > uschind...@apache.org > ASF Member, Apache Lucene PMC / Committer > Bremen, Germany > http://lucene.apache.org/ > > > -Original Message- > > From: Andrew Bayer [mailto:andrew.ba...@gmail.com] > > Sent: Thursday, August 20, 2015 7:33 PM > > To: builds@apache.org > > Subject: FYI - disabled timer triggers across the board on > builds.apache.org > > > > I noticed a lot of jobs that were running every day regardless of whether > > anything changed and eating up a lot of executors, so I bulk-removed all > of > > them, changing those jobs to poll for changes hourly instead. If your > project > > has one or two jobs that you need to run daily whether there are code > > changes or not, you can re-enable the timer, but please do not do that > for > > more than a couple jobs, and please do not do it for jobs that take > longer > > than half an hour. > > > > A. > >
Re: Name resolution issue on ubuntu jenkins slaves?
Using google public dns should be fine. Removed yahoo in colo resolver, 67.195.2.197 which is accessible from hosted apache nodes. DHCPD configs is also updated to issue revolvers as 8.8.8.8/8.8.4.4 . -rajive From: Andrew Bayer To: "builds@apache.org" Cc: Keith W ; Rajiv Chittajallu Sent: Thursday, August 20, 2015 10:23 AM Subject: Re: Name resolution issue on ubuntu jenkins slaves? FYI, I've manually overwritten /etc/resolv.conf on all the H* and ubuntu-* slaves with something more valid. A. On Thu, Aug 20, 2015 at 1:16 PM, Andrew Bayer wrote: Yup, http://drewgraybeal.blogspot.com/2015/05/level-3-dns-hijacking-4222-and-others.html - level3 is hijacking NXDOMAIN now. I think Y!'s DHCP is setting those DNS servers? Rajiv, can you comment? A. On Thu, Aug 20, 2015 at 10:32 AM, Andrew Bayer wrote: So the Yahoo! provided slaves (H*, ubuntu-*) do have Level3's DNS servers in /etc/resolv.conf - we don't actually control that directly, it's part of the system setup Y! uses, I think. I'll ping our contact at Y! about changing them to use more standard DNS. A. On Thu, Aug 20, 2015 at 6:15 AM, Gavin McDonald wrote: > On 20 Aug 2015, at 4:03 am, David Nalley wrote: > > Whatever we are using for DNS on those build slaves is redirecting any > NXDOMAIN to a Level3 search domain. > > Is this happening on the ubuntu-* slaves or on the dynamic build slaves? It was mentioned earlier : > I am certainly seeing this on slaves ubuntu4 > through ubuntu6. I have seen passes and failures on the dynamic slaves too, so seems inconsistent. Gav… > > --David > > On Mon, Aug 10, 2015 at 8:59 AM, Keith W wrote: >> Hello Apache builds, >> >> We (Apache Qpid) have been seeing a test fail on the Ubuntu Jenkins slaves >> since August 9th. The test verifies the behaviour of the code when the >> user specifies a hostname that does not exist in DNS. For this purpose, >> the test uses a random name 'hg3sgaaw4lgihjs' (without hierarchal part) >> which is assumed not resolve. This test is longstanding and has been >> running on the slaves for many years without issue. >> >> Between August 9th and 10th, something appears to have changed on the >> slaves, which is meaning that the lookup of the name is now returning an >> IP. This is causing the Java test to fail. I've investigated by >> introducing shell commands into the job, and can see evidence of the same >> problem at the UNIX level. I am certainly seeing this on slaves ubuntu4 >> through ubuntu6. >> >> >> $ host hg3sgaaw4lgihjs >> hg3sgaaw4lgihjs has address 198.105.244.11 >> hg3sgaaw4lgihjs has address 198.105.254.11 >> Host hg3sgaaw4lgihjs not found: 3(NXDOMAIN) >> >> $ host hg3sgaaw4lgihjs2 >> hg3sgaaw4lgihjs2 has address 198.105.244.11 >> hg3sgaaw4lgihjs2 has address 198.105.254.11 >> Host hg3sgaaw4lgihjs2 not found: 3(NXDOMAIN) >> >> >> I considered changing the test to use a RFC-2606 Reserved Top Level DNS >> Names hg3sgaaw4lgihjs.invalid but I notice that it too is resolving to >> 198.105.244.11 too. The fact that an .invalid address is resolving makes >> me suspect there is an environmental problem at the root cause. >> >> host hg3sgaaw4lgihjs.invalid >> hg3sgaaw4lgihjs.invalid has address 198.105.244.11 >> hg3sgaaw4lgihjs.invalid has address 198.105.254.11 >> Host hg3sgaaw4lgihjs.invalid not found: 3(NXDOMAIN) >> >> Is there a name resolution issue affecting these hosts? >> >> Example job affected by the issue: >> >> https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-Test-JDK1.8/ >> >> Kind regards, Keith Wall.
