On Mon, Feb 17, 2020 at 5:28 PM Wes McKinney <wesmck...@gmail.com> wrote: > > On Mon, Feb 17, 2020 at 10:19 AM Neal Richardson > <neal.p.richard...@gmail.com> wrote: > > > > Ok, I made https://issues.apache.org/jira/browse/ARROW-7870 to look into > > this further. > > > > If the flaky nightly builds persist, maybe we should suspend the uploading > > to GitHub releases until we sort it out? The noise will block out real > > failures in the nightlies that we need to address. > > Yeah, that seems reasonable. These are the exact same Crossbow tasks > that are used for releases, so we'll need some option to toggle off > the artifact uploads in the nightly job.
In order to test that the artifact upload is working we should upload the assets from nightly jobs as well. I find it pretty strange that the uploads are rejected, because it used to work much more stable than other alternatives (like the builtin travis deployer). Perhaps github has changed something because this issue was around previously https://github.com/sigmavirus24/github3.py/issues/779#issuecomment-379470626 Created a blocker jira to track it https://issues.apache.org/jira/browse/ARROW-7877 > > > Neal > > > > On Mon, Feb 17, 2020 at 8:08 AM Wes McKinney <wesmck...@gmail.com> wrote: > > > > > On Mon, Feb 17, 2020 at 9:57 AM Neal Richardson > > > <neal.p.richard...@gmail.com> wrote: > > > > > > > > Maybe the new nightly wheel GitHub Page is triggering rate limits, as > > > > Krisztián suggested it might: https://github.com/apache/arrow/pull/6366. > > > > The increase in timeouts seems correlated with when that PR merged. > > > > > > > > I wonder if there's a better solution to making nightly wheels > > > > available. > > > > GitHub releases (on a crossbow repo separate from apache/arrow, no less) > > > > seem like an unnatural fit for the problem. > > > > > > Agreed. I think we can make nightly development builds available in a > > > more formalized way as long as there are appropriate warning signs > > > that they should not be depended on in production, see e.g. another > > > Apache project giving links to nightlies > > > > > > https://jmeter.apache.org/nightly.html > > > > > > > Neal > > > > > > > > > > > > On Mon, Feb 17, 2020 at 3:35 AM Wes McKinney <wesmck...@gmail.com> > > > wrote: > > > > > > > > > It seems like GitHub release uploads continue to be a bit flaky. Is > > > > > there a JIRA about adding some retry logic with delay (e.g. waiting > > > > > 30-60 seconds in between tries) to attempt to reduce the flakiness? > > > > > > > > > > > > > https://github.com/apache/arrow/blob/master/dev/tasks/crossbow.py#L1517 > > > > > > > > > > On Sun, Feb 16, 2020 at 7:31 AM Crossbow <cross...@ursalabs.org> > > > wrote: > > > > > > > > > > > > > > > > > > Arrow Build Report for Job nightly-2020-02-16-0 > > > > > > > > > > > > All tasks: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0 > > > > > > > > > > > > Failed Tasks: > > > > > > - centos-7: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-centos-7 > > > > > > - conda-linux-gcc-py27: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-linux-gcc-py27 > > > > > > - conda-osx-clang-py27: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-osx-clang-py27 > > > > > > - debian-stretch: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-debian-stretch > > > > > > - gandiva-jar-trusty: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-gandiva-jar-trusty > > > > > > - test-conda-python-2.7-pandas-latest: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-2.7-pandas-latest > > > > > > - test-conda-python-2.7: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-2.7 > > > > > > - test-conda-python-3.7-pandas-master: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-pandas-master > > > > > > - test-conda-python-3.7-spark-master: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-spark-master > > > > > > - test-conda-python-3.7-turbodbc-latest: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-turbodbc-latest > > > > > > - test-conda-python-3.7-turbodbc-master: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-turbodbc-master > > > > > > - test-ubuntu-18.04-docs: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-docs > > > > > > - ubuntu-bionic: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-ubuntu-bionic > > > > > > - ubuntu-disco: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-ubuntu-disco > > > > > > - wheel-manylinux1-cp35m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux1-cp35m > > > > > > - wheel-manylinux1-cp36m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux1-cp36m > > > > > > > > > > > > Succeeded Tasks: > > > > > > - centos-6: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-centos-6 > > > > > > - centos-8: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-centos-8 > > > > > > - conda-linux-gcc-py36: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-linux-gcc-py36 > > > > > > - conda-linux-gcc-py37: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-linux-gcc-py37 > > > > > > - conda-linux-gcc-py38: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-linux-gcc-py38 > > > > > > - conda-osx-clang-py36: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-osx-clang-py36 > > > > > > - conda-osx-clang-py37: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-osx-clang-py37 > > > > > > - conda-osx-clang-py38: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-osx-clang-py38 > > > > > > - conda-win-vs2015-py36: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-win-vs2015-py36 > > > > > > - conda-win-vs2015-py37: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-win-vs2015-py37 > > > > > > - conda-win-vs2015-py38: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-win-vs2015-py38 > > > > > > - debian-buster: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-debian-buster > > > > > > - gandiva-jar-osx: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-gandiva-jar-osx > > > > > > - homebrew-cpp: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-homebrew-cpp > > > > > > - macos-r-autobrew: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-macos-r-autobrew > > > > > > - test-conda-cpp-valgrind: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-cpp-valgrind > > > > > > - test-conda-cpp: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-cpp > > > > > > - test-conda-python-3.6: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.6 > > > > > > - test-conda-python-3.7-dask-latest: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-dask-latest > > > > > > - test-conda-python-3.7-hdfs-2.9.2: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-hdfs-2.9.2 > > > > > > - test-conda-python-3.7-pandas-latest: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-pandas-latest > > > > > > - test-conda-python-3.7: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7 > > > > > > - test-conda-python-3.8-dask-master: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.8-dask-master > > > > > > - test-conda-python-3.8-pandas-latest: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.8-pandas-latest > > > > > > - test-conda-r-3.6: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-r-3.6 > > > > > > - test-debian-10-cpp: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-debian-10-cpp > > > > > > - test-debian-10-go-1.12: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-debian-10-go-1.12 > > > > > > - test-debian-10-python-3: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-debian-10-python-3 > > > > > > - test-debian-c-glib: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-debian-c-glib > > > > > > - test-debian-ruby: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-debian-ruby > > > > > > - test-fedora-29-cpp: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-fedora-29-cpp > > > > > > - test-fedora-29-python-3: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-fedora-29-python-3 > > > > > > - test-r-rhub-debian-gcc-devel: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rhub-debian-gcc-devel > > > > > > - test-r-rhub-ubuntu-gcc-release: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rhub-ubuntu-gcc-release > > > > > > - test-r-rstudio-r-base-3.6-bionic: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rstudio-r-base-3.6-bionic > > > > > > - test-r-rstudio-r-base-3.6-centos6: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rstudio-r-base-3.6-centos6 > > > > > > - test-r-rstudio-r-base-3.6-opensuse15: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rstudio-r-base-3.6-opensuse15 > > > > > > - test-r-rstudio-r-base-3.6-opensuse42: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rstudio-r-base-3.6-opensuse42 > > > > > > - test-ubuntu-16.04-cpp: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-16.04-cpp > > > > > > - test-ubuntu-18.04-cpp-cmake32: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-cpp-cmake32 > > > > > > - test-ubuntu-18.04-cpp-release: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-cpp-release > > > > > > - test-ubuntu-18.04-cpp-static: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-cpp-static > > > > > > - test-ubuntu-18.04-cpp: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-cpp > > > > > > - test-ubuntu-18.04-python-3: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-python-3 > > > > > > - test-ubuntu-18.04-r-sanitizer: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-r-sanitizer > > > > > > - test-ubuntu-c-glib: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-c-glib > > > > > > - test-ubuntu-ruby: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-ruby > > > > > > - ubuntu-eoan: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-ubuntu-eoan > > > > > > - ubuntu-xenial: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-ubuntu-xenial > > > > > > - wheel-manylinux1-cp37m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux1-cp37m > > > > > > - wheel-manylinux1-cp38: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux1-cp38 > > > > > > - wheel-manylinux2010-cp35m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2010-cp35m > > > > > > - wheel-manylinux2010-cp36m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2010-cp36m > > > > > > - wheel-manylinux2010-cp37m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2010-cp37m > > > > > > - wheel-manylinux2010-cp38: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2010-cp38 > > > > > > - wheel-manylinux2014-cp35m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2014-cp35m > > > > > > - wheel-manylinux2014-cp36m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2014-cp36m > > > > > > - wheel-manylinux2014-cp37m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2014-cp37m > > > > > > - wheel-manylinux2014-cp38: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2014-cp38 > > > > > > - wheel-osx-cp35m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-wheel-osx-cp35m > > > > > > - wheel-osx-cp36m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-wheel-osx-cp36m > > > > > > - wheel-osx-cp37m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-wheel-osx-cp37m > > > > > > - wheel-osx-cp38: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-wheel-osx-cp38 > > > > > > - wheel-win-cp36m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-appveyor-wheel-win-cp36m > > > > > > - wheel-win-cp37m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-appveyor-wheel-win-cp37m > > > > > > - wheel-win-cp38: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-appveyor-wheel-win-cp38 > > > > > > > >
