Terry J. Reedy <tjre...@udel.edu> added the comment:
I think this test disabling and failure issue should get some attention.
Do the failures happen on non-Windows systems too?
With installed 3.3.0b1 on Win7, test___all__, test_tcl, test_tk,
test_ttk_textonly, and test_ttk_guionly all run if run first (_tk and
_ttk_guionly with -ugui). All modify os.environ, resulting in
Warning -- os.environ was modified by test_xxx
and all disable at least one of the other tests run afterward. (test___all__
disables but is not disabled itself.) In standard alphabetic running order,
test___all__ 'wins'.
I would like to add test/test_idle, #15392, but I presume it currently would
also not run after test___all__ or any of the others.
---
I patched my installation, but without the unneeded temporary (saved_values =
self.saved_values is only an optimization for the repeated access in the loop
that follows).
sys.modules.clear()
sys.modules.update(self.saved_modules)
del self.saved_modules
With this,
python -m test -ugui test___all__ test_tcl test_tk test_ttk_guionly
test_ttk_textonly
runs and passes all five tests instead of two (test___all__ and
test_ttk_textonly).
Unfortunately, it causes new failures running the entire test suite:
test_concurrent_futures (ok before)
test_decimal (ok before)
test_email (numberous failures, just one before)
I stopped at test_faulthandler, there might be more but this was enough to
reject the patch as is.
I decided to try the principle of doing the minimum necessary to get the tests
to pass. I arrived at the following, which lets all five tests run and pass, at
least when run in the given order.
try:
del sys.modules['tkinter']
del sys.modules['tkinter._fix']
del sys.modules['tkinter.ttk']
del sys.modules['tkinter.test']
del sys.modules['tkinter.test.support']
del sys.modules['tkinter.test.runtktests']
except KeyError:
pass
The only new failure with python -m test is test_multi-processing, which
normally does not even give a warning.
[197/368/6] test_multiprocessing
Warning -- multiprocessing.process._dangling was modified by
test_multiprocessing
test test_multiprocessing failed -- Traceback (most recent call last):
File "C:\Programs\Python33\lib\test\test_multiprocessing.py", line 1909, in
test_mymanager_context_prestarted
self.assertEqual(manager._process.exitcode, 0)
AssertionError: -15 != 0
This might be an unrelated, intermittant failure.
Since the exception above will trigger for nearly all tests (costly), and since
the above does not guarantee to delete all tkinter.* modules, and since future
tkinter tests might expand to test other subpackages and idle tests will import
other subpackages, I tried this in __exit__.
if 'tkinter' in sys.modules:
sysmod = sys.modules
tknames = set()
for name in sysmod:
if name.startswith('tkinter'):
tknames.add(name)
for name in tknames:
del sysmod[name]
So aside from the initial test, the extra time is only used when needed. All
five tests run and pass and there are no new failures with python -m test.
Before:
[368/368/7] test_zlib
316 tests OK.
7 tests failed:
test_asyncore test_datetime test_distutils test_email test_ftplib
test_import test_tcl
2 tests altered the execution environment:
test___all__ test_builtin
43 tests skipped:
test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp
test_codecmaps_kr test_codecmaps_tw test_crypt test_curses
test_dbm_gnu test_dbm_ndbm test_devpoll test_epoll test_fcntl
test_fork1 test_gdb test_grp test_ioctl test_kqueue test_largefile
test_nis test_openpty test_ossaudiodev test_pipes test_poll
test_posix test_pty test_pwd test_readline test_resource
test_smtpnet test_socketserver test_syslog test_threadsignals
test_timeout test_tk test_tools test_ttk_guionly test_urllib2net
test_urllibnet test_wait3 test_wait4 test_winsound test_xmlrpc_net
test_zipfile64
5 skips unexpected on win32:
test_gdb test_readline test_tk test_tools test_ttk_guionly
After:
315 tests OK.
6 tests failed:
test_asyncore test_datetime test_distutils test_email test_ftplib
test_import
4 tests altered the execution environment:
test___all__ test_builtin test_tcl test_ttk_textonly
43 tests skipped:
test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp
test_codecmaps_kr test_codecmaps_tw test_crypt test_curses
test_dbm_gnu test_dbm_ndbm test_devpoll test_epoll test_fcntl
test_fork1 test_gdb test_grp test_ioctl test_kqueue test_largefile
test_nis test_openpty test_ossaudiodev test_pipes test_poll
test_posix test_pty test_pwd test_readline test_resource
test_smtpnet test_socketserver test_syslog test_threadsignals
test_timeout test_tk test_tools test_ttk_guionly test_urllib2net
test_urllibnet test_wait3 test_wait4 test_winsound test_xmlrpc_net
test_zipfile64
3 skips unexpected on win32:
test_gdb test_readline test_tools
I am puzzled why one less failure and two fewer unexpected skips means one few
test ok. Is that because of the two more warnings?
Running with -ugui will cause test_tk and test_ttk_guionly to run instead of
skip. So the above patch seems to be a pure win on Windows. What about other
systems? And do any of you test experts have anything better?
----------
nosy: +ezio.melotti, michael.foord, pitrou, r.david.murray, terry.reedy
stage: -> patch review
type: -> behavior
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue10652>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
