[issue25894] unittest subTest failure causes result to be omitted from listing

Łukasz Langa added the comment: > Any suggestions for the format of output? Currently PR 28082 formats lines > for subtest skipping, failure or error with a 2-space identation. Lines for > skipping, failure or error in tearDown() or functions registered with > addCallback() do not differ from

[issue25894] unittest subTest failure causes result to be omitted from listing

Łukasz Langa added the comment: New changeset f0f29f328d8b4568e8c0d4c55c7d120d96f80911 by Serhiy Storchaka in branch 'main': bpo-25894: Always report skipped and failed subtests separately (GH-28082) https://github.com/python/cpython/commit/f0f29f328d8b4568e8c0d4c55c7d120d96f80911 --

[issue25894] unittest subTest failure causes result to be omitted from listing

Serhiy Storchaka added the comment: Any suggestions for the format of output? Currently PR 28082 formats lines for subtest skipping, failure or error with a 2-space identation. Lines for skipping, failure or error in tearDown() or functions registered with addCallback() do not differ from a

[issue25894] unittest subTest failure causes result to be omitted from listing

Serhiy Storchaka added the comment: PR 28082 is a draft that implements this idea. Skipped and failed (but not successfully passed) subtests are now reported separately, as a character (sFE) or a line ("skipped", "FAIL", "ERROR"). The description of the subtest is included in the line. For e

[issue25894] unittest subTest failure causes result to be omitted from listing

Change by Serhiy Storchaka : -- keywords: +patch pull_requests: +26525 stage: test needed -> patch review pull_request: https://github.com/python/cpython/pull/28082 ___ Python tracker

[issue25894] unittest subTest failure causes result to be omitted from listing

Serhiy Storchaka added the comment: Were not subtests proposed as a more flexible replacement of parametrized tests? I think that every subtest should be counted as a separate test case: in verbose mode it should output a separate line and in non-verbose mode it should output a separate char

[issue25894] unittest subTest failure causes result to be omitted from listing

Robert Collins added the comment: The basic model is this: - a test can have a single outcome [yes, the api is ambiguous, but there it is] - subtests let you identify multiple variations of a single test (note the id tweak etc) and *may* be reported differently We certainly must not report th

[issue25894] unittest subTest failure causes result to be omitted from listing

Ezio Melotti added the comment: > My gut feeling is that any subtest failure should be counted as the > whole test failing. I’m not sure how the failure vs error cases should > be handled. Maybe error should trump failure. I think the priority should be error > failure > skip > pass. IOW, pass s

[issue25894] unittest subTest failure causes result to be omitted from listing

Martin Panter added the comment: Yes now I understand. If a subtest fails, there is no status update (not even a newline in verbose mode), and each subtest skip triggers a separate status update. My gut feeling is that any subtest failure should be counted as the whole test failing. I’m not s

[issue25894] unittest subTest failure causes result to be omitted from listing

R. David Murray added the comment: I believe this was discussed at the time subTest was added and deemed an acceptable tradeoff for a simpler implementation. I'm not sure it is, but I'm not prepared to write code to fix it :) I'm bothered every time I see this, but I have to admit that the t

[issue25894] unittest subTest failure causes result to be omitted from listing

Zachary Ware added the comment: Martin Panter added the comment: > Okay, so you have a test with subtests. You have presented three cases: > > 1. Single subtest which passes. No problem I assume. Or several subtests which pass. No problems. > 2. Two subtests: 1st fails, 2nd passes. This is how

[issue25894] unittest subTest failure causes result to be omitted from listing

Martin Panter added the comment: Okay, so you have a test with subtests. You have presented three cases: 1. Single subtest which passes. No problem I assume. 2. Two subtests: 1st fails, 2nd passes. This is how subtests are normally used, so I guess there is no problem. Is that right? 3. After

[issue25894] unittest subTest failure causes result to be omitted from listing

New submission from Zachary Ware: The title can barely be called accurate; the description of the problem isn't easy to condense to title length. Here's the issue: $ cat subtest_test.py import os import unittest class TestClass(unittest.TestCase): def test_subTest(self): for t i