New submission from STINNER Victor <victor.stin...@haypocalc.com>: Sporadic issues are difficult to analyze on the buildbots because sometimes a test fails, but when it runs again in verbose mode... it doesn't fail anymore. Typical example:
=============================== [ 99/356/1] test_smtplib Re-running test test_smtplib in verbose mode (...) testTimeoutValue (test.test_smtplib.GeneralTests) ... ok testBasic (test.test_smtplib.DebuggingServerTests) ... test test_smtplib failed -- multiple errors occurred; run in verbose mode for details ok testHELP (test.test_smtplib.DebuggingServerTests) ... ok testNOOP (test.test_smtplib.DebuggingServerTests) ... ok testNotImplemented (test.test_smtplib.DebuggingServerTests) ... ok (...) ---------------------------------------------------------------------- Ran 32 tests in 23.664s OK =============================== I propose to always run the tests in verbose mode, write the output into a buffer, but only display the output on failure (or ignore the output on success). Attached patch simplifies regrtest.py usage: replace 4 verbose levels (no verbose option, -v, -w, -W) by only 2 (no verbose option, -v). Always display the full output in case of an error (as we used -w before): no more "multiple error occured" without output trap, no more "oh, rerunning the test doesn't fail" trap. The -v option works as before (directly write the output in verbose mode). regrtest is much more verbose than before in case of a failure, but I prefer too much information than no information when I try to debug something. If it is too much verbose by default, I can try to use 3 verbose levels (e.g. don't display the full output if only one test failed). If the patch is accepted, the devguide should also be patched. ---------- components: Tests files: regrtest_verbose.patch keywords: patch messages: 138981 nosy: ezio.melotti, georg.brandl, haypo, pitrou, r.david.murray priority: normal severity: normal status: open title: regrtest: always run tests in verbose mode, but hide the output on success versions: Python 2.7, Python 3.2, Python 3.3 Added file: http://bugs.python.org/file22446/regrtest_verbose.patch _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue12400> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
