Re-sending to openstack-infra mailing list
From: Amrith Kumar Sent: Tuesday, November 29, 2016 9:16 AM To: Monty Taylor <monty.tay...@gmail.com>; jer...@openstack.org; 'clark.boy...@gmail.com' <clark.boy...@gmail.com>; 'Andreas Jaeger' <a...@suse.com>; 'Paul Belanger' <pabelan...@redhat.com>; 'yrobl...@redhat.com' <yrobl...@redhat.com> Cc: Peter Stachowski <pe...@tesora.com>; 'Matt Riedemann' <mrie...@us.ibm.com>; 'dansm...@redhat.com' <dansm...@redhat.com> Subject: [infra] some issue with the osic environment? (as opposed to rax) Importance: High We've been trying to get all of the trove CI tests to work with neutron and have made a lot of progress but are now hampered by something that appears to be out of our control. Please see the latest (say, patch set 52) of review 356701 [1]. 1. Ignore all the Ubuntu-xenial jobs, that is a work in progress at this point and the failures are known 2. Please only focus on the jobs gate-trove-functional-dsvm-mysql-ubuntu-trusty and gate-trove-scenario-dsvm-* -ubuntu-trusty What we find is that if the job gets scheduled on rax, it works. If it gets scheduled on osic, it fails. The failure mode indicates that when building a guest image, we're not able to get to the internet; something is messed up in the routing on the machine. How and why this works on rax and not on osic, I know not but this appears to be out of our control. Failure (from: http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mysq l-ubuntu-trusty/b9e5197/console.html) 2016-11-29 11:44:42.309943 <http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_309943> | 2016-11-29 11:44:42.309 | ++ /tmp/dib_build.7kbGxUCC/hooks/bin/cache-url:main:89 : eval curl -v -L -o /home/jenkins/.cache/image-create/source-repositories/.download.JYWwOK94 -w '%{http_code}' https://bootstrap.pypa.io/get-pip.py 2016-11-29 11:44:42.315614 <http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_315614> | 2016-11-29 11:44:42.315 | +++ /tmp/dib_build.7kbGxUCC/hooks/bin/cache-url:main:89 : curl -v -L -o /home/jenkins/.cache/image-create/source-repositories/.download.JYWwOK94 -w '%{http_code}' https://bootstrap.pypa.io/get-pip.py 2016-11-29 11:44:42.317559 <http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_317559> | 2016-11-29 11:44:42.317 | * Hostname was NOT found in DNS cache 2016-11-29 11:44:42.318830 <http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_318830> | 2016-11-29 11:44:42.318 | % Total % Received % Xferd Average Speed Time Time Time Current 2016-11-29 11:44:42.320197 <http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_320197> | 2016-11-29 11:44:42.319 | Dload Upload Total Spent Left Speed 2016-11-29 11:44:42.442269 <http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_442269> | 2016-11-29 11:44:42.442 | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 151.101.48.175... 2016-11-29 11:46:49.726905 <http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_46_49_726905> | 2016-11-29 11:46:49.724 | 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0 [.] 0 0 0 0 0 0 0 0 --:--:-- 0:02:06 --:--:-- 0* connect to 151.101.48.175 port 443 failed: Connection timed out 2016-11-29 11:46:49.728470 <http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_46_49_728470> | 2016-11-29 11:46:49.728 | * Failed to connect to bootstrap.pypa.io port 443: Connection timed out 2016-11-29 11:46:49.730314 <http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_46_49_730314> | 2016-11-29 11:46:49.730 | * Closing connection 0 2016-11-29 11:46:49.732029 <http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_46_49_732029> | 2016-11-29 11:46:49.731 | curl: (7) Failed to connect to bootstrap.pypa.io port 443: Connection timed out Success (from: http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-si ngle-ubuntu-trusty/3837026/console.html) 2016-11-29 12:04:16.795230 <http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_795230> | 2016-11-29 12:04:16.794 | + /tmp/dib_build.zLErDYEt/hooks/bin/cache-url:main:82 : '[' -f /home/jenkins/.cache/image-create/source-repositories/pip_and_virtualenv_50b bba586670cca2803e23f614374b482e595f6a -a -s /home/jenkins/.cache/image-create/source-repositories/pip_and_virtualenv_50b bba586670cca2803e23f614374b482e595f6a ']' 2016-11-29 12:04:16.798157 <http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_798157> | 2016-11-29 12:04:16.797 | + /tmp/dib_build.zLErDYEt/hooks/bin/cache-url:main:86 : success='Downloaded and cached https://bootstrap.pypa.io/get-pip.py for the first time' 2016-11-29 12:04:16.802066 <http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_802066> | 2016-11-29 12:04:16.801 | ++ /tmp/dib_build.zLErDYEt/hooks/bin/cache-url:main:89 : eval curl -v -L -o /home/jenkins/.cache/image-create/source-repositories/.download.LGdrOCXI -w '%{http_code}' https://bootstrap.pypa.io/get-pip.py 2016-11-29 12:04:16.805188 <http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_805188> | 2016-11-29 12:04:16.804 | +++ /tmp/dib_build.zLErDYEt/hooks/bin/cache-url:main:89 : curl -v -L -o /home/jenkins/.cache/image-create/source-repositories/.download.LGdrOCXI -w '%{http_code}' https://bootstrap.pypa.io/get-pip.py 2016-11-29 12:04:16.812077 <http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_812077> | 2016-11-29 12:04:16.811 | * Hostname was NOT found in DNS cache 2016-11-29 12:04:16.813773 <http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_813773> | 2016-11-29 12:04:16.813 | % Total % Received % Xferd Average Speed Time Time Time Current 2016-11-29 12:04:16.815340 <http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_815340> | 2016-11-29 12:04:16.815 | Dload Upload Total Spent Left Speed 2016-11-29 12:04:16.937806 <http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_937806> | 2016-11-29 12:04:16.937 | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 151.101.44.175... 2016-11-29 12:04:16.939302 <http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_939302> | 2016-11-29 12:04:16.939 | * Connected to bootstrap.pypa.io (151.101.44.175) port 443 (#0) Would someone in infra please help us figure out what's going on here please. We are currently NOT depending on Monty's change (398611). Should we be? If yes, what's the ETA for a merge there? Thanks, -amrith [1] https://review.openstack.org/#/c/356701/52 [2] https://review.openstack.org/#/c/398611/ -- Amrith Kumar +1-978-563-9590 amr...@tesora.com <mailto:amr...@amrith.org> http://www.tesora.com/ GPG: 0x5E48849A9D21A29B
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
