Ok, I made https://issues.apache.org/jira/browse/ARROW-7870 to look into this further.
If the flaky nightly builds persist, maybe we should suspend the uploading to GitHub releases until we sort it out? The noise will block out real failures in the nightlies that we need to address. Neal On Mon, Feb 17, 2020 at 8:08 AM Wes McKinney <wesmck...@gmail.com> wrote: > On Mon, Feb 17, 2020 at 9:57 AM Neal Richardson > <neal.p.richard...@gmail.com> wrote: > > > > Maybe the new nightly wheel GitHub Page is triggering rate limits, as > > Krisztián suggested it might: https://github.com/apache/arrow/pull/6366. > > The increase in timeouts seems correlated with when that PR merged. > > > > I wonder if there's a better solution to making nightly wheels available. > > GitHub releases (on a crossbow repo separate from apache/arrow, no less) > > seem like an unnatural fit for the problem. > > Agreed. I think we can make nightly development builds available in a > more formalized way as long as there are appropriate warning signs > that they should not be depended on in production, see e.g. another > Apache project giving links to nightlies > > https://jmeter.apache.org/nightly.html > > > Neal > > > > > > On Mon, Feb 17, 2020 at 3:35 AM Wes McKinney <wesmck...@gmail.com> > wrote: > > > > > It seems like GitHub release uploads continue to be a bit flaky. Is > > > there a JIRA about adding some retry logic with delay (e.g. waiting > > > 30-60 seconds in between tries) to attempt to reduce the flakiness? > > > > > > > https://github.com/apache/arrow/blob/master/dev/tasks/crossbow.py#L1517 > > > > > > On Sun, Feb 16, 2020 at 7:31 AM Crossbow <cross...@ursalabs.org> > wrote: > > > > > > > > > > > > Arrow Build Report for Job nightly-2020-02-16-0 > > > > > > > > All tasks: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0 > > > > > > > > Failed Tasks: > > > > - centos-7: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-centos-7 > > > > - conda-linux-gcc-py27: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-linux-gcc-py27 > > > > - conda-osx-clang-py27: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-osx-clang-py27 > > > > - debian-stretch: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-debian-stretch > > > > - gandiva-jar-trusty: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-gandiva-jar-trusty > > > > - test-conda-python-2.7-pandas-latest: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-2.7-pandas-latest > > > > - test-conda-python-2.7: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-2.7 > > > > - test-conda-python-3.7-pandas-master: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-pandas-master > > > > - test-conda-python-3.7-spark-master: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-spark-master > > > > - test-conda-python-3.7-turbodbc-latest: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-turbodbc-latest > > > > - test-conda-python-3.7-turbodbc-master: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-turbodbc-master > > > > - test-ubuntu-18.04-docs: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-docs > > > > - ubuntu-bionic: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-ubuntu-bionic > > > > - ubuntu-disco: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-ubuntu-disco > > > > - wheel-manylinux1-cp35m: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux1-cp35m > > > > - wheel-manylinux1-cp36m: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux1-cp36m > > > > > > > > Succeeded Tasks: > > > > - centos-6: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-centos-6 > > > > - centos-8: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-centos-8 > > > > - conda-linux-gcc-py36: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-linux-gcc-py36 > > > > - conda-linux-gcc-py37: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-linux-gcc-py37 > > > > - conda-linux-gcc-py38: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-linux-gcc-py38 > > > > - conda-osx-clang-py36: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-osx-clang-py36 > > > > - conda-osx-clang-py37: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-osx-clang-py37 > > > > - conda-osx-clang-py38: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-osx-clang-py38 > > > > - conda-win-vs2015-py36: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-win-vs2015-py36 > > > > - conda-win-vs2015-py37: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-win-vs2015-py37 > > > > - conda-win-vs2015-py38: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-win-vs2015-py38 > > > > - debian-buster: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-debian-buster > > > > - gandiva-jar-osx: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-gandiva-jar-osx > > > > - homebrew-cpp: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-homebrew-cpp > > > > - macos-r-autobrew: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-macos-r-autobrew > > > > - test-conda-cpp-valgrind: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-cpp-valgrind > > > > - test-conda-cpp: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-cpp > > > > - test-conda-python-3.6: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.6 > > > > - test-conda-python-3.7-dask-latest: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-dask-latest > > > > - test-conda-python-3.7-hdfs-2.9.2: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-hdfs-2.9.2 > > > > - test-conda-python-3.7-pandas-latest: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-pandas-latest > > > > - test-conda-python-3.7: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7 > > > > - test-conda-python-3.8-dask-master: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.8-dask-master > > > > - test-conda-python-3.8-pandas-latest: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.8-pandas-latest > > > > - test-conda-r-3.6: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-r-3.6 > > > > - test-debian-10-cpp: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-debian-10-cpp > > > > - test-debian-10-go-1.12: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-debian-10-go-1.12 > > > > - test-debian-10-python-3: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-debian-10-python-3 > > > > - test-debian-c-glib: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-debian-c-glib > > > > - test-debian-ruby: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-debian-ruby > > > > - test-fedora-29-cpp: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-fedora-29-cpp > > > > - test-fedora-29-python-3: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-fedora-29-python-3 > > > > - test-r-rhub-debian-gcc-devel: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rhub-debian-gcc-devel > > > > - test-r-rhub-ubuntu-gcc-release: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rhub-ubuntu-gcc-release > > > > - test-r-rstudio-r-base-3.6-bionic: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rstudio-r-base-3.6-bionic > > > > - test-r-rstudio-r-base-3.6-centos6: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rstudio-r-base-3.6-centos6 > > > > - test-r-rstudio-r-base-3.6-opensuse15: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rstudio-r-base-3.6-opensuse15 > > > > - test-r-rstudio-r-base-3.6-opensuse42: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rstudio-r-base-3.6-opensuse42 > > > > - test-ubuntu-16.04-cpp: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-16.04-cpp > > > > - test-ubuntu-18.04-cpp-cmake32: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-cpp-cmake32 > > > > - test-ubuntu-18.04-cpp-release: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-cpp-release > > > > - test-ubuntu-18.04-cpp-static: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-cpp-static > > > > - test-ubuntu-18.04-cpp: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-cpp > > > > - test-ubuntu-18.04-python-3: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-python-3 > > > > - test-ubuntu-18.04-r-sanitizer: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-r-sanitizer > > > > - test-ubuntu-c-glib: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-c-glib > > > > - test-ubuntu-ruby: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-ruby > > > > - ubuntu-eoan: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-ubuntu-eoan > > > > - ubuntu-xenial: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-ubuntu-xenial > > > > - wheel-manylinux1-cp37m: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux1-cp37m > > > > - wheel-manylinux1-cp38: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux1-cp38 > > > > - wheel-manylinux2010-cp35m: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2010-cp35m > > > > - wheel-manylinux2010-cp36m: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2010-cp36m > > > > - wheel-manylinux2010-cp37m: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2010-cp37m > > > > - wheel-manylinux2010-cp38: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2010-cp38 > > > > - wheel-manylinux2014-cp35m: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2014-cp35m > > > > - wheel-manylinux2014-cp36m: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2014-cp36m > > > > - wheel-manylinux2014-cp37m: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2014-cp37m > > > > - wheel-manylinux2014-cp38: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2014-cp38 > > > > - wheel-osx-cp35m: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-wheel-osx-cp35m > > > > - wheel-osx-cp36m: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-wheel-osx-cp36m > > > > - wheel-osx-cp37m: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-wheel-osx-cp37m > > > > - wheel-osx-cp38: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-wheel-osx-cp38 > > > > - wheel-win-cp36m: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-appveyor-wheel-win-cp36m > > > > - wheel-win-cp37m: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-appveyor-wheel-win-cp37m > > > > - wheel-win-cp38: > > > > URL: > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-appveyor-wheel-win-cp38 > > > >
