New submission from STINNER Victor:
The Lib/test/regrtest.py file became a monster. It's very long, it uses a lot
of global variables. It's hard to review and hard to maintain. I propose to:
- split single file regrtest.py into multiple files in a new Lib/test/regrtest/
package
- refactor the test runner as a class to avoid global variables
- refactor the multiprocessing test runner as a class in a separated file
Attached patch implements this idea.
The following commands still work:
./python -m test [options] [args]
./python -m test.regrtest [options] [args]
But the following command doesn't work anymore:
./python Lib/test/regrtest.py [options] [args]
If regrtest.py is executed directly on buildbots, these buildbots should be
modified first.
The "./python -m test.regrtest" command should work on all Python versions
(Python 2 and Python 3) and so should be preferred. The "./python -m test"
command only works on Python 3 (thanks to the new __main__.py feature).
The change adds a new feature: it now displays name of concurrent tests running
since 30 seconds or longer when the multiprocessing test runner is used (-j
command line option). Example:
...
[240/399] test_uu
[241/399] test_urllib_response
[242/399] test_struct
[243/399] test_descrtut
[244/399] test_threadedtempfile
[245/399] test_tracemalloc -- running: test_concurrent_futures (30 sec)
[246/399] test_dbm_dumb -- running: test_concurrent_futures (30 sec)
[247/399] test_codeop -- running: test_concurrent_futures (30 sec)
...
[395/399/1] test_asyncio -- running: test_multiprocessing_fork (40 sec),
test_multiprocessing_spawn (44 sec)
[396/399/1] test_faulthandler -- running: test_multiprocessing_fork (50
sec), test_multiprocessing_spawn (54 sec)
[397/399/1] test_multiprocessing_fork (52 sec) -- running:
test_multiprocessing_spawn (56 sec)
[398/399/1] test_multiprocessing_spawn (68 sec) -- running:
test_multiprocessing_forkserver (39 sec)
[399/399/1] test_multiprocessing_forkserver (50 sec)
I want this feature to analysis why more and more buildbots fail with a timeout
without saying which test was running (well, I suspect multiprocessing
tests...).
Note: faulthandler can show where regrtest is blocked, but not when the
multiprocessing test runner is used. And sometimes the process is killed by the
buildbot, not by faulthandler :-/
Another minor new feature: on CTRL+c, it shows which tests are running when the
multiprocessing test runner is used. Example:
[ 38/399] test_dummy_thread
[ 39/399] test_codecmaps_jp
[ 40/399] test_future5
^C
Waiting for test_scope, test_decimal, test_memoryview, test_heapq,
test_unicodedata, test_trace, test_threadsignals, test_cgitb, test_runpy,
test_cmd_line_script
Other changes:
* Show test timing when a test runs longer than 30 seconds
* Don't make __file__ absolute, findtestdir() calls os.path.abspath() instead.
Remove these lines:
__file__ = os.path.abspath(__file__)
assert __file__ == os.path.abspath(sys.argv[0])
* print() is now called wih flush=True (it didn't check all calls, only the
major calls), remove sys.stdout.flush() and sys.stdout.flush()
* A lot of refactoring. Sorry, I didn't take notes for each change.
I fear that test_regrtest has a small code coverage. I only tested major
options, I didn't test -R for example.
Note: I don't understand how the --single option works when regrtest is not run
from the Python source code directory. A temporary directory, so the pynexttest
file is removed after its creation, no? If it doesn't make sense to use
--single outside Python directory, maybe an error should be raised?
----------
components: Tests
files: regrtest_package.patch
keywords: patch
messages: 251419
nosy: ezio.melotti, haypo, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Enhance and refactor test.regrtest (convert regrtest.py to a package)
versions: Python 3.6
Added file: http://bugs.python.org/file40554/regrtest_package.patch
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25220>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
