Hello!

In Kubernetes, we have extensive infrastructure for collecting test 
failures. For "go test" based tests (unit tests, integration tests) we use 
gotest.tools/gotestsum to convert the output to a JUnit file for 
post-processing.

This approach has one drawback 
(https://github.com/kubernetes/kubernetes/issues/116402): the entire output 
of a failed tests is treated as the (in JUnit terminology) failure message. 
In particular for integration tests that also includes normal log output. 
The tool which triages failures then fails to detect that all of these test 
runs failed for the same reason because the "failure message" is too 
different for each run.

My initial reaction was that we should modify our tests so that each 
`t.Error` call uses a message with a unique prefix ("[FAILED]"). Then we 
could identify those failures during post-processing.

But this is going to be cumbersome to do and enforce, in particular when 
there are additional third-party helpers involved. It would be much easier 
if we could do `go test -failure-prefix [FAILED]` and then get that prefix 
injected automatically into all calls which combine logging with `t.Fail`. 
Would something like this be acceptable?

I'm aware that a test could do `t.Log("I failed."); t.Fail()`. But this 
should be rare and could be treated like it is now by the post-processing 
(i.e. failure without a specific failure message).

Bye, Patrick

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0d5780d2-f1ec-448d-b3d9-7ed05ee9d5adn%40googlegroups.com.

Reply via email to