An alternative - we can cache our python dependencies (via pip wheel) in svn.apache.org under 3rdparty and disable PyPI lookups (as they add a flaky external dependency to our tests).
On Mon, Oct 20, 2014 at 1:00 PM, Zameer Manji <zma...@twopensource.com> wrote: > It seems that we need to add HTTP retries to pex > <https://github.com/pantsbuild/pex/issues/18> or use another system to > resolve and download dependencies. > > On Mon, Oct 20, 2014 at 12:58 PM, Apache Jenkins Server < > jenk...@builds.apache.org> wrote: > > > See <https://builds.apache.org/job/Aurora/666/changes> > > > > Changes: > > > > [mchucarroll] Improve error messages in client commands. > > > > ------------------------------------------ > > [...truncated 87 lines...] > > ++ which python2.7 > > + PY=/usr/bin/python2.7 > > + echo 'Using /usr/bin/python2.7' > > Using /usr/bin/python2.7 > > +++ dirname < > > https://builds.apache.org/job/Aurora/ws/build-support/virtualenv> > > ++ cd <https://builds.apache.org/job/Aurora/ws/build-support> > > ++ pwd > > + HERE=<https://builds.apache.org/job/Aurora/ws/build-support> > > + '[' -f < > > > https://builds.apache.org/job/Aurora/ws/build-support/virtualenv-1.10.1/BOOTSTRAPPED > > > > ']' > > + pushd <https://builds.apache.org/job/Aurora/ws/build-support> > > ~/jenkins-slave/workspace/Aurora/build-support > > ~/jenkins-slave/workspace/Aurora > > + curl -O > > > https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.10.1.tar.gz > > % Total % Received % Xferd Average Speed Time Time Time > > Current > > Dload Upload Total Spent Left > > Speed > > > > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- > > 0 > > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- > > 0 > > 17 1294k 17 224k 0 0 160k 0 0:00:08 0:00:01 0:00:07 > > 160k > > 100 1294k 100 1294k 0 0 628k 0 0:00:02 0:00:02 --:--:-- > > 629k > > + tar zxvf virtualenv-1.10.1.tar.gz > > virtualenv-1.10.1/ > > virtualenv-1.10.1/AUTHORS.txt > > virtualenv-1.10.1/bin/ > > virtualenv-1.10.1/bin/rebuild-script.py > > virtualenv-1.10.1/docs/ > > virtualenv-1.10.1/docs/_theme/ > > virtualenv-1.10.1/docs/_theme/nature/ > > virtualenv-1.10.1/docs/_theme/nature/static/ > > virtualenv-1.10.1/docs/_theme/nature/static/nature.css_t > > virtualenv-1.10.1/docs/_theme/nature/static/pygments.css > > virtualenv-1.10.1/docs/_theme/nature/theme.conf > > virtualenv-1.10.1/docs/conf.py > > virtualenv-1.10.1/docs/index.rst > > virtualenv-1.10.1/docs/make.bat > > virtualenv-1.10.1/docs/Makefile > > virtualenv-1.10.1/docs/news.rst > > virtualenv-1.10.1/LICENSE.txt > > virtualenv-1.10.1/MANIFEST.in > > virtualenv-1.10.1/PKG-INFO > > virtualenv-1.10.1/README.rst > > virtualenv-1.10.1/scripts/ > > virtualenv-1.10.1/scripts/virtualenv > > virtualenv-1.10.1/setup.cfg > > virtualenv-1.10.1/setup.py > > virtualenv-1.10.1/virtualenv.egg-info/ > > virtualenv-1.10.1/virtualenv.egg-info/dependency_links.txt > > virtualenv-1.10.1/virtualenv.egg-info/entry_points.txt > > virtualenv-1.10.1/virtualenv.egg-info/not-zip-safe > > virtualenv-1.10.1/virtualenv.egg-info/PKG-INFO > > virtualenv-1.10.1/virtualenv.egg-info/SOURCES.txt > > virtualenv-1.10.1/virtualenv.egg-info/top_level.txt > > virtualenv-1.10.1/virtualenv.py > > virtualenv-1.10.1/virtualenv_embedded/ > > virtualenv-1.10.1/virtualenv_embedded/activate.bat > > virtualenv-1.10.1/virtualenv_embedded/activate.csh > > virtualenv-1.10.1/virtualenv_embedded/activate.fish > > virtualenv-1.10.1/virtualenv_embedded/activate.ps1 > > virtualenv-1.10.1/virtualenv_embedded/activate.sh > > virtualenv-1.10.1/virtualenv_embedded/activate_this.py > > virtualenv-1.10.1/virtualenv_embedded/deactivate.bat > > virtualenv-1.10.1/virtualenv_embedded/distutils-init.py > > virtualenv-1.10.1/virtualenv_embedded/distutils.cfg > > virtualenv-1.10.1/virtualenv_embedded/site.py > > virtualenv-1.10.1/virtualenv_support/ > > virtualenv-1.10.1/virtualenv_support/__init__.py > > virtualenv-1.10.1/virtualenv_support/pip-1.4.1.tar.gz > > virtualenv-1.10.1/virtualenv_support/setuptools-0.9.8.tar.gz > > + touch virtualenv-1.10.1/BOOTSTRAPPED > > + popd > > ~/jenkins-slave/workspace/Aurora > > + exec /usr/bin/python2.7 < > > > https://builds.apache.org/job/Aurora/ws/build-support/virtualenv-1.10.1/virtualenv.py > > > > <https://builds.apache.org/job/Aurora/ws/build-support/pex.venv> > > New python executable in < > > > https://builds.apache.org/job/Aurora/ws/build-support/pex.venv/bin/python2.7 > > > > > Also creating executable in < > > > https://builds.apache.org/job/Aurora/ws/build-support/pex.venv/bin/python> > > Installing > > > Setuptools..............................................................................................................................................................................................................................done. > > Installing > > > Pip.....................................................................................................................................................................................................................................................................................................................................done. > > Downloading/unpacking twitter.common.python==0.5.9 > > Running setup.py egg_info for package twitter.common.python > > > > Downloading/unpacking setuptools>=1 (from twitter.common.python==0.5.9) > > Running setup.py egg_info for package setuptools > > > > Downloading/unpacking wheel (from twitter.common.python==0.5.9) > > Running setup.py egg_info for package wheel > > > > no previously-included directories found matching 'wheel/test/*/dist' > > no previously-included directories found matching > 'wheel/test/*/build' > > Installing collected packages: twitter.common.python, setuptools, wheel > > Running setup.py install for twitter.common.python > > Skipping installation of < > > > https://builds.apache.org/job/Aurora/ws/build-support/pex.venv/lib/python2.7/site-packages/twitter/__init__.py > > > > (namespace package) > > Skipping installation of < > > > https://builds.apache.org/job/Aurora/ws/build-support/pex.venv/lib/python2.7/site-packages/twitter/common/__init__.py > > > > (namespace package) > > > > Installing < > > > https://builds.apache.org/job/Aurora/ws/build-support/pex.venv/lib/python2.7/site-packages/twitter.common.python-0.5.9-py2.7-nspkg.pth > > > > > Installing pex script to < > > https://builds.apache.org/job/Aurora/ws/build-support/pex.venv/bin> > > Found existing installation: setuptools 0.9.8 > > Uninstalling setuptools: > > Successfully uninstalled setuptools > > Running setup.py install for setuptools > > > > Installing easy_install script to < > > https://builds.apache.org/job/Aurora/ws/build-support/pex.venv/bin> > > Installing easy_install-2.7 script to < > > https://builds.apache.org/job/Aurora/ws/build-support/pex.venv/bin> > > Running setup.py install for wheel > > > > no previously-included directories found matching 'wheel/test/*/dist' > > no previously-included directories found matching > 'wheel/test/*/build' > > Installing wheel script to < > > https://builds.apache.org/job/Aurora/ws/build-support/pex.venv/bin> > > Successfully installed twitter.common.python setuptools wheel > > Cleaning up... > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 0.8ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 1.0ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 1.0ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 0.7ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 0.7ms > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.process/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.process/ :: Opening > > (uncached) > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.process/ :: Opening > > (uncached) :: Fetching > > https://pypi.python.org/simple/twitter.common.process/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.process/ :: Opening > > (uncached) :: Fetching > > https://pypi.python.org/simple/twitter.common.process/ :: Resolving > > pypi.python.org > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.process/ :: Opening > > (uncached) :: Fetching > > https://pypi.python.org/simple/twitter.common.process/ :: Connecting to > > pypi.python.org > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.process/ :: Opening > > (uncached) :: Caching > > https://pypi.python.org/simple/twitter.common.process/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.process/: 559.7ms > > twitter.common.python.http: Opening (uncached): 558.5ms > > twitter.common.python.http: Fetching > > https://pypi.python.org/simple/twitter.common.process/: 557.5ms > > twitter.common.python.http: Resolving pypi.python.org: 64.3ms > > twitter.common.python.http: Connecting to pypi.python.org: 134.1ms > > twitter.common.python.http: Caching > > https://pypi.python.org/simple/twitter.common.process/: 0.6ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 1.2ms > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.log/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.log/ :: Opening (uncached) > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.log/ :: Opening (uncached) > > :: Fetching https://pypi.python.org/simple/twitter.common.log/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.log/ :: Opening (uncached) > > :: Fetching https://pypi.python.org/simple/twitter.common.log/ :: > > Resolving pypi.python.org > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.log/ :: Opening (uncached) > > :: Fetching https://pypi.python.org/simple/twitter.common.log/ :: > > Connecting to pypi.python.org > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.log/ :: Opening (uncached) > > :: Caching https://pypi.python.org/simple/twitter.common.log/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.log/: 559.1ms > > twitter.common.python.http: Opening (uncached): 558.4ms > > twitter.common.python.http: Fetching > > https://pypi.python.org/simple/twitter.common.log/: 557.5ms > > twitter.common.python.http: Resolving pypi.python.org: 64.3ms > > twitter.common.python.http: Connecting to pypi.python.org: 135.7ms > > twitter.common.python.http: Caching > > https://pypi.python.org/simple/twitter.common.log/: 0.4ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 0.8ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 0.8ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 0.7ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 0.6ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 0.6ms > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/pytest-cov/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/pytest-cov/ :: Opening (uncached) > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/pytest-cov/ :: Opening (uncached) :: > > Fetching https://pypi.python.org/simple/pytest-cov/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/pytest-cov/ :: Opening (uncached) :: > > Fetching https://pypi.python.org/simple/pytest-cov/ :: Resolving > > pypi.python.org > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/pytest-cov/ :: Opening (uncached) :: > > Fetching https://pypi.python.org/simple/pytest-cov/ :: Connecting to > > pypi.python.org > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/pytest-cov/ :: Opening (uncached) :: > > Caching https://pypi.python.org/simple/pytest-cov/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/pytest-cov/: 551.9ms > > twitter.common.python.http: Opening (uncached): 550.2ms > > twitter.common.python.http: Fetching > > https://pypi.python.org/simple/pytest-cov/: 549.2ms > > twitter.common.python.http: Resolving pypi.python.org: 64.2ms > > twitter.common.python.http: Connecting to pypi.python.org: 131.3ms > > twitter.common.python.http: Caching > > https://pypi.python.org/simple/pytest-cov/: 0.5ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 0.5ms > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.util/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.util/ :: Opening > (uncached) > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.util/ :: Opening > (uncached) > > :: Fetching https://pypi.python.org/simple/twitter.common.util/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.util/ :: Opening > (uncached) > > :: Fetching https://pypi.python.org/simple/twitter.common.util/ :: > > Resolving pypi.python.org > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.util/ :: Opening > (uncached) > > :: Fetching https://pypi.python.org/simple/twitter.common.util/ :: > > Connecting to pypi.python.org > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.util/ :: Opening > (uncached) > > :: Caching https://pypi.python.org/simple/twitter.common.util/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.util/: 596.9ms > > twitter.common.python.http: Opening (uncached): 596.2ms > > twitter.common.python.http: Fetching > > https://pypi.python.org/simple/twitter.common.util/: 595.3ms > > twitter.common.python.http: Resolving pypi.python.org: 114.3ms > > twitter.common.python.http: Connecting to pypi.python.org: 128.0ms > > twitter.common.python.http: Caching > > https://pypi.python.org/simple/twitter.common.util/: 0.5ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 0.8ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 1.0ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 0.7ms > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.lang/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.lang/ :: Opening > (uncached) > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.lang/ :: Opening > (uncached) > > :: Fetching https://pypi.python.org/simple/twitter.common.lang/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.lang/ :: Opening > (uncached) > > :: Fetching https://pypi.python.org/simple/twitter.common.lang/ :: > > Resolving pypi.python.org > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.lang/ :: Opening > (uncached) > > :: Fetching https://pypi.python.org/simple/twitter.common.lang/ :: > > Connecting to pypi.python.org > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.lang/ :: Opening > (uncached) > > :: Caching https://pypi.python.org/simple/twitter.common.lang/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.lang/: 5488.1ms > > twitter.common.python.http: Opening (uncached): 5487.4ms > > twitter.common.python.http: Fetching > > https://pypi.python.org/simple/twitter.common.lang/: 5486.4ms > > twitter.common.python.http: Resolving pypi.python.org: 64.2ms > > twitter.common.python.http: Connecting to pypi.python.org: 131.3ms > > twitter.common.python.http: Caching > > https://pypi.python.org/simple/twitter.common.lang/: 0.5ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 0.7ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 0.7ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 0.6ms > > twitter.common.python.http: Crawling https://pypi.python.org/simple/pex/ > > twitter.common.python.http: Crawling https://pypi.python.org/simple/pex/ > > :: Opening (uncached) > > twitter.common.python.http: Crawling https://pypi.python.org/simple/pex/ > > :: Opening (uncached) :: Fetching https://pypi.python.org/simple/pex/ > > twitter.common.python.http: Crawling https://pypi.python.org/simple/pex/ > > :: Opening (uncached) :: Fetching https://pypi.python.org/simple/pex/ :: > > Resolving pypi.python.org > > twitter.common.python.http: Crawling https://pypi.python.org/simple/pex/ > > :: Opening (uncached) :: Fetching https://pypi.python.org/simple/pex/ :: > > Connecting to pypi.python.org > > twitter.common.python.http: Crawling https://pypi.python.org/simple/pex/ > > :: Opening (uncached) :: Caching https://pypi.python.org/simple/pex/ > > twitter.common.python.http: Crawling https://pypi.python.org/simple/pex/ > : > > 561.8ms > > twitter.common.python.http: Opening (uncached): 561.2ms > > twitter.common.python.http: Fetching > > https://pypi.python.org/simple/pex/: 560.3ms > > twitter.common.python.http: Resolving pypi.python.org: 64.1ms > > twitter.common.python.http: Connecting to pypi.python.org: 126.2ms > > twitter.common.python.http: Caching > > https://pypi.python.org/simple/pex/: 0.5ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 0.8ms > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.config/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.config/ :: Opening > > (uncached) > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.config/ :: Opening > > (uncached) :: Fetching > > https://pypi.python.org/simple/twitter.common.config/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.config/ :: Opening > > (uncached) :: Fetching > > https://pypi.python.org/simple/twitter.common.config/ :: Resolving > > pypi.python.org > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.config/ :: Opening > > (uncached) :: Fetching > > https://pypi.python.org/simple/twitter.common.config/ :: Connecting to > > pypi.python.org > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.config/ :: Opening > > (uncached) :: Caching > > https://pypi.python.org/simple/twitter.common.config/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/twitter.common.config/: 567.8ms > > twitter.common.python.http: Opening (uncached): 567.0ms > > twitter.common.python.http: Fetching > > https://pypi.python.org/simple/twitter.common.config/: 566.0ms > > twitter.common.python.http: Resolving pypi.python.org: 64.2ms > > twitter.common.python.http: Connecting to pypi.python.org: 132.0ms > > twitter.common.python.http: Caching > > https://pypi.python.org/simple/twitter.common.config/: 0.6ms > > twitter.common.python.http: Crawling /home/jenkins/.pex/build > > twitter.common.python.http: Crawling /home/jenkins/.pex/build: 0.7ms > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/pytest/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/pytest/ :: Opening (uncached) > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/pytest/ :: Opening (uncached) :: Fetching > > https://pypi.python.org/simple/pytest/ > > twitter.common.python.http: Crawling > > https://pypi.python.org/simple/pytest/ :: Opening (uncached) :: Fetching > > https://pypi.python.org/simple/pytest/ :: Resolving pypi.python.org > > twitter.common.python.http: Timed out resolving pypi.python.org > > twitter.common.python.http: Failed to fetch > > https://pypi.python.org/simple/pytest/: Could not reach > > https://pypi.python.org/simple/pytest/ within deadline. > > Traceback (most recent call last): > > File "< > > https://builds.apache.org/job/Aurora/ws/build-support/pex.venv/bin/pex > ",> > > line 9, in <module> > > load_entry_point('twitter.common.python==0.5.9', 'console_scripts', > > 'pex')() > > File "< > > > https://builds.apache.org/job/Aurora/ws/build-support/pex.venv/local/lib/python2.7/site-packages/twitter/common/python/bin/pex.py > ",> > > line 320, in main > > pex_builder = build_pex(args, options) > > File "< > > > https://builds.apache.org/job/Aurora/ws/build-support/pex.venv/local/lib/python2.7/site-packages/twitter/common/python/bin/pex.py > ",> > > line 284, in build_pex > > platform=options.platform) > > File "< > > > https://builds.apache.org/job/Aurora/ws/build-support/pex.venv/local/lib/python2.7/site-packages/twitter/common/python/resolver.py > ",> > > line 98, in resolve > > raise Unsatisfiable('Cannot satisfy requirements: %s' % > > requirement_set[requirement.key]) > > twitter.common.python.resolver.Unsatisfiable: Cannot satisfy > requirements: > > [Requirement.parse('pytest>=2.6,<2.7')] > > Build step 'Execute shell' marked build as failure > > Recording test results > > > > > > -- > Zameer Manji >
