[Python-Dev] Recent buildbot reports and asyncio test failures
Hi everyone, Just a heads-up regarding some messages you will see in your pull requests. There is an intermittent failure on some buildbots regarding asyncio: https://buildbot.python.org/all/#/builders/21 As the builds do not fail all the time, the systems understand that if your (merged) commits fail to build, they may be the cause of the failure and then it does a report into the pull request. I am working on investigating a way to improve the report mechanism to make it less noisy in this case, but bear in mind that the correct way to solve this is fixing the asyncio bug in the test suite and this won't likely go away completely until is solved. We are doing all that we can to solve all the recent leaks and failures on the test suite, but there is a noticeable increase in the number of merged pull requests because of the imminent feature freeze and because this happens across several timezones is very difficult to get them all. Thanks to everyone that is helping solving these bugs :) Regards from sunny London, Pablo Galindo Salgado ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Recent buildbot reports and asyncio test failures
Many tests are failing randomly on buildbots. *MANY* failures: "test_asyncio: test_cancel_gather_2() dangling thread" https://bugs.python.org/issue37137 Some failures on specific buildbots. multiprocessing does crash randomly on Windows: https://bugs.python.org/issue37143 and on FreeBSD: https://bugs.python.org/issue37135 I bet on a regression caused by: https://bugs.python.org/issue33608#msg344240 One failure. "test_asyncio timed out on AMD64 FreeBSD CURRENT Shared 3.x" https://bugs.python.org/issue37142 I failed to reproduce any of these bugs. Victor Le lun. 3 juin 2019 à 12:42, Pablo Galindo Salgado a écrit : > > Hi everyone, > > Just a heads-up regarding some messages you will see in your pull requests. > There is an intermittent failure on some buildbots > regarding asyncio: > > https://buildbot.python.org/all/#/builders/21 > > As the builds do not fail all the time, the systems understand that if your > (merged) commits fail to build, they may be the cause > of the failure and then it does a report into the pull request. > > I am working on investigating a way to improve the report mechanism to make > it less noisy in this case, but bear in mind that > the correct way to solve this is fixing the asyncio bug in the test suite and > this won't likely go away completely until is solved. > > We are doing all that we can to solve all the recent leaks and failures on > the test suite, but there is a noticeable increase in the > number of merged pull requests because of the imminent feature freeze and > because this happens across several timezones > is very difficult to get them all. > > Thanks to everyone that is helping solving these bugs :) > > Regards from sunny London, > Pablo Galindo Salgado > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Recent buildbot reports and asyncio test failures
Hi, Update. I'm collaborating with Pablo to fix these recent regressions. We had to revert 2 changes to fix asyncio and multiprocessing test failures and crashes. New issue: "opcode cache for LOAD_GLOBAL emits false alarm in memory leak hunting" https://bugs.python.org/issue37146 INADA-san worked around this issue by disabling the new opcode cache (LOAD_GLOBAL optimization) when Python is compiled in debug mode. We should try to find a more long term solution after beta1 release. Le lun. 3 juin 2019 à 17:05, Victor Stinner a écrit : > *MANY* failures: "test_asyncio: test_cancel_gather_2() dangling thread" > https://bugs.python.org/issue37137 Andrew Svetlov identified the root issue, it's a side effect of a new feature: https://bugs.python.org/issue37137#msg344488 He reverted his change to fix the bug. We may reconsider the feature after beta1 (that's up to our Release Manager, I guess). Note: I pushed 2 changes to attempt to fix the issue, but they didn't fix the issue. We can consider to revert them after beta1. > Some failures on specific buildbots. multiprocessing does crash > randomly on Windows: >https://bugs.python.org/issue37143 > and on FreeBSD: >https://bugs.python.org/issue37135 > I bet on a regression caused by: >https://bugs.python.org/issue33608#msg344240 Pablo and me identified the root issue: https://bugs.python.org/issue37135#msg344509 I reverted a change to fix the regression. Sadly, previously I also reverted a different change which wasn't the root cause. We can consider to maybe reapply it after beta1. (!) "Night gathers, and now my watch begins. It shall not end until my death." (!) Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
