Ezio Melotti <ezio.melo...@gmail.com> added the comment:
IMHO the current "unexpected" list is fairly pointless. I just ran the test
suite on 3.3 and I got the usual expected list of unexpected skips:
7 skips unexpected on linux2:
test_bz2 test_dbm_gnu test_dbm_ndbm test_tcl test_tk
test_ttk_guionly test_ttk_textonly
If these tests were really *expected to run* on my machine, having them skipped
would be a failure. They are instead just a set of tests that I *might be able
to run* once I install all the right dependencies.
I think the tests can be classified in 3 categories:
1) tests that are known to always run everywhere (e.g. test_unittest);
2) tests that are known to always run on specific platforms and never on
others (e.g. test_winreg);
3) tests that might run on some platforms depending on some specific condition
(e.g. test_zlib);
For each of these cases, regrtest should:
1) fail if the test is not run;
2) fail if the test is not run on the platform(s) where it's always available,
always skip it on the others;
3) skip the test conditionally without failing (e.g. checking if zlib can be
imported or not);
A special case of 3) is when the skip depends on the platform name rather than
the availability of some specific module/function (e.g. if some platforms
shipped with a buggy version of a library and there's no way to check if the
library is buggy other than keeping a list of platforms that are known to use
that version). In this case the condition should check only for the platform
where it's known to fail, and more platforms can be added if/when people report
them.
I agree that the logic for this should be moved out of regrtest and added to
the specific tests.
It's also possible to provide better skip messages, e.g.:
-test_winreg skipped -- No module named 'winreg'
-test_tk skipped -- No module named '_tkinter'
-test_urllibnet skipped -- Use of the `network' resource not enabled
+test_winreg skipped -- No module named 'winreg' (winreg is available on
windows only)
+test_tk skipped -- No module named '_tkinter' (tcl/tk not installed)
+test_urllibnet skipped -- Use of the `network' resource not enabled (run the
tests with -unetwork or -uall to enable it)
----------
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue10966>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
