Re: [OpenStack-Infra] Deep pipelining with zuul

2016-02-24 Thread Jeremy Stanley
On 2016-02-24 09:19:49 -0800 (-0800), Jerry Zhao wrote: > I remember that there is a delay for used nodes on nodepool to be cleaned, > without zuul marking it used, new jobs could be assigned on the used node, > which would be deleted later so new job would sure fail. There is an OFFLINE_NODE_WHEN

Re: [OpenStack-Infra] Deep pipelining with zuul

2016-02-24 Thread Jerry Zhao
I remember that there is a delay for used nodes on nodepool to be cleaned, without zuul marking it used, new jobs could be assigned on the used node, which would be deleted later so new job would sure fail. On 02/24/2016 08:58 AM, Doug Wiegley wrote: I’m doing something similar, but am just sp

Re: [OpenStack-Infra] Deep pipelining with zuul

2016-02-24 Thread Andrew Grimberg
I could do that, but I don't get the auto scaling that nodepool offers since nodepool only inspects the zuul queue for needs and not the Jenkins build queue. Unless there's a way to make nodepool monitor the Jenkins build queue as well that I'm not aware of. We're operating in a public cloud so we

Re: [OpenStack-Infra] Deep pipelining with zuul

2016-02-24 Thread Doug Wiegley
I’m doing something similar, but am just specifying nodepool nodes on the child jobs, with artifact copying in Jenkins. I don’t think you need JClouds at that point. Jobs that use nodepool nodes don’t have to originate with zuul. Or am I missing your restriction? doug > On Feb 24, 2016, at 10

Re: [OpenStack-Infra] Deep pipelining with zuul

2016-02-24 Thread Andrew Grimberg
The problem with that is that we would then continue to have to rely on JClouds to do slave management and we have two driving reasons for this desire / need to switch to Zuul and Nodepool (or something similar) 1) Recent versions of the Gerrit Trigger plugin for Jenkins forget, or do not update c

Re: [OpenStack-Infra] Deep pipelining with zuul

2016-02-24 Thread Doug Wiegley
Even if zuul doesn’t support the flow you want, you can also still use matrix, multijob, and downstream jobs on Jenkins itself to simulate the same behavior, with zuul just monitoring the parent job. Mix and match until you get the flow that you need. doug > On Feb 24, 2016, at 9:56 AM, Andre

[OpenStack-Infra] Deep pipelining with zuul

2016-02-24 Thread Andrew Grimberg
Greetings, We're working on transitioning our Jenkins system off of Gerrit Trigger and JClouds to Zuul and Nodepool and have run into a bit of a snag. We have some rather deep pipeline dependencies with our current configurations where one job produces information and possibly artifacts that are