As another data point, from Wednesday to Friday last week rh1 ran at
full capacity pretty much round the clock. There were experimental jobs
that queued for at least 18 hours.
Granted, this is a symptom of a capacity problem we've exacerbated by
adding the containers OVB job, but even without that the experimental
jobs generally weren't getting run until after US working hours, at
which point we're only a few hours from the regular periodic pipeline run.
On 03/19/2017 11:54 AM, Sagi Shnaidman wrote:
Hi, Paul
I would say that real worthwhile try starts from "normal" priority,
because we want to run promotion jobs more *often*, not more *rarely*
which happens with low priority.
In addition the initial idea in the first mail was running them each
after other almost, not once a day like it happens now or with "low"
priority.
Thanks
On Wed, Mar 15, 2017 at 11:16 PM, Paul Belanger <pabelan...@redhat.com
<mailto:pabelan...@redhat.com>> wrote:
On Wed, Mar 15, 2017 at 03:42:32PM -0500, Ben Nemec wrote:
>
>
> On 03/13/2017 02:29 PM, Sagi Shnaidman wrote:
> > Hi, all
> >
> > I submitted a change: https://review.openstack.org/#/c/443964/
<https://review.openstack.org/#/c/443964/>
> > but seems like it reached a point which requires an additional
discussion.
> >
> > I had a few proposals, it's increasing period to 12 hours
instead of 4
> > for start, and to leave it in regular periodic *low* precedence.
> > I think we can start from 12 hours period to see how it goes,
although I
> > don't think that 4 only jobs will increase load on OVB cloud, it's
> > completely negligible comparing to current OVB capacity and load.
> > But making its precedence as "low" IMHO completely removes any sense
> > from this pipeline to be, because we already run
experimental-tripleo
> > pipeline which this priority and it could reach timeouts like 7-14
> > hours. So let's assume we ran periodic job, it's queued to run
now 12 +
> > "low queue length" - about 20 and more hours. It's even worse
than usual
> > periodic job and definitely makes this change useless.
> > I'd like to notice as well that those periodic jobs unlike "usual"
> > periodic are used for repository promotion and their value are
equal or
> > higher than check jobs, so it needs to run with "normal" or even
"high"
> > precedence.
>
> Yeah, it makes no sense from an OVB perspective to add these as
low priority
> jobs. Once in a while we've managed to chew through the entire
experimental
> queue during the day, but with the containers job added it's very
unlikely
> that's going to happen anymore. Right now we have a 4.5 hour wait
time just
> for the check queue, then there's two hours of experimental jobs
queued up
> behind that. All of which means if we started a low priority
periodic job
> right now it probably wouldn't run until about midnight my time,
which I
> think is when the regular periodic jobs run now.
>
Lets just give it a try? A 12 hour periodic job with low priority.
There is
nothing saying we cannot iterate on this after a few days / weeks /
months.
> >
> > Thanks
> >
> >
> > On Thu, Mar 9, 2017 at 10:06 PM, Wesley Hayutin
<whayu...@redhat.com <mailto:whayu...@redhat.com>
> > <mailto:whayu...@redhat.com <mailto:whayu...@redhat.com>>> wrote:
> >
> >
> >
> > On Wed, Mar 8, 2017 at 1:29 PM, Jeremy Stanley
<fu...@yuggoth.org <mailto:fu...@yuggoth.org>
> > <mailto:fu...@yuggoth.org <mailto:fu...@yuggoth.org>>> wrote:
> >
> > On 2017-03-07 10:12:58 -0500 (-0500), Wesley Hayutin wrote:
> > > The TripleO team would like to initiate a conversation
about the
> > > possibility of creating a new pipeline in Openstack
Infra to allow
> > > a set of jobs to run periodically every four hours
> > [...]
> >
> > The request doesn't strike me as
contentious/controversial. Why not
> > just propose your addition to the zuul/layout.yaml file
in the
> > openstack-infra/project-config repo and hash out any
resulting
> > concerns via code review?
> > --
> > Jeremy Stanley
> >
> >
> > Sounds good to me.
> > We thought it would be nice to walk through it in an email
first :)
> >
> > Thanks
> >
> >
> >
__________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> >
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
> >
<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>>
> >
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>>
> >
> >
> >
> >
__________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> >
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
> >
<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>>
> >
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> >
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>>
> >
> >
> >
> >
> > --
> > Best regards
> > Sagi Shnaidman
> >
> >
> >
__________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
> >
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> >
>
>
__________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
--
Best regards
Sagi Shnaidman
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
