Re: [openstack-dev] [openstack-ansible] Debugging slow Xenial gate

2016-11-10 Thread Major Hayden
On 11/02/2016 08:51 AM, Major Hayden wrote: > At this point, I'm still trying to test some additional theories. Does anyone > have any other ideas? Here's an update for today. There are a few bugs open now: OpenStack-Ansible bug: https://bugs.launchpad.net/openstack-ansible/+bug/1637494 Ub

Re: [openstack-dev] [openstack-ansible] Debugging slow Xenial gate

2016-11-02 Thread Jesse Pretorius
> On 11/2/16, 1:51 PM, "Major Hayden" wrote: >I tossed up a horribly written hack[0] to change some CPU scheduler > settings back to the Trusty settings. My initial tests were great! Also, > the first test in OpenStack CI was really good -- 62 minutes for trusty and > 65 minutes for xeni

Re: [openstack-dev] [openstack-ansible] Debugging slow Xenial gate

2016-11-02 Thread Major Hayden
On 10/28/2016 04:02 AM, Major Hayden wrote: > On the topic of threads, the sysbench output from both Trusty and Xenial are > nearly identical with the exception of threads. Trusty is usually about > 15-20% faster on that benchmark than Xenial. I spoke with a few other people and it seems like t

Re: [openstack-dev] [openstack-ansible] Debugging slow Xenial gate

2016-10-28 Thread Major Hayden
On 10/28/2016 10:17 AM, Major Hayden wrote: >> Also, when running the tests on both systems, track cpu usage and number >> > of threads to see if one has more restrictions than the other. > Almost no difference here. On the topic of threads, the sysbench output from both Trusty and Xenial are nea

Re: [openstack-dev] [openstack-ansible] Debugging slow Xenial gate

2016-10-28 Thread Major Hayden
On 10/28/2016 01:44 AM, Mike Carden wrote: > I bounced this off my 'distro differences' goto guy, Chris Smart. Here are > his thoughts: > > "Run the 14.04 kernel on 16.04 system and re-run the tests to see if it's > kernel related. > > If 16.04 userland with 14.04 kernel is as fast as Ubuntu 14.

Re: [openstack-dev] [openstack-ansible] Debugging slow Xenial gate

2016-10-27 Thread Mike Carden
Major, I bounced this off my 'distro differences' goto guy, Chris Smart. Here are his thoughts: "Run the 14.04 kernel on 16.04 system and re-run the tests to see if it's kernel related. If 16.04 userland with 14.04 kernel is as fast as Ubuntu 14.04, then compare the kernel .config files to see i

[openstack-dev] [openstack-ansible] Debugging slow Xenial gate

2016-10-27 Thread Major Hayden
Hey there, We've talked about the slow Xenial gate during the OpenStack Summit this week and I decided to do a little digging. I built two quick test instances: one with Trusty and the other with Xenial. Trusty comes with python 2.7.6 and Xenial has 2.7.12. Here are the initial comparisons: