On 6/30/25 16:41, Soham wrote:
Hello,
I recently was using the tap driver provided by automake, and noticed
that there currently is no way to declare a test as xfail or xpass
apart from marking it as "todo". The test ouput however does produce
rows for XPASS and XFAIL, and the existing TRS format already supports
both.
Since the TAP specification allows for custom directives [1], I was
wondering if there was interest in a patch adding these to
`lib/tap-driver.sh`. I would add "XPASS" and "XFAIL" directives, and
if the test did not pass/fail as expected, it would result in an ERROR.
Perhaps Autotest uses ERROR differently, but the better way to think
about it is expected-vs.-unexpected results. A test marked XFAIL that
fails is expected to fail, but a test marked XFAIL that passes is a
surprise XPASS.
A better solution is to add a directive that declares that certain tests
are expected to fail. The TAP driver then matches the actual results
with the expected results.
-- Jacob
