Hi Maxime, Maxime Devos <maximede...@telenet.be> writes:
>>> Based on this I believe it describes the specification. >> >>That's correct. It's been slightly modified in places where it said >>things like "left to the implementation" and I was able to verify what >>the current implementation in Guix does. > > I assume Guix->Guile. > > This modification of “left to the implementation” -> “what Guile does” is > problematic, since it misleads readers into thinking this is the standard > behaviour (it is after all named SRFI 64, not GRFI 64). > > “What Guile does” is also important information to have. Yes, I agree. I think this is actually what I did, but my way to wording of this above was not very good, and my memory fails. A diff of a freshly "snarfed" [0] srfi-64.html (from its most recent commit) against the documented submitted, which was hand-tuned:
diff -u /tmp/srfi-64-resnarfed.texi /tmp/srfi-64-documented.texi
--- /tmp/srfi-64-resnarfed.texi 2024-10-02 15:57:27.562406299 +0900
+++ /tmp/srfi-64-documented.texi 2024-10-02 15:58:57.798218571 +0900
@@ -1,71 +1,58 @@
-@node SRFI 64
-@subsection SRFI 64: A Scheme API for test suites
-@cindex SRFI 64
-
-
@node SRFI 64 Abstract
@subsubsection SRFI 64 Abstract
-
-This defines an API for writing @dfn{test suites}, to make it easy
-to portably test Scheme APIs, libraries, applications, and implementations.
-A test suite is a collection of @dfn{test cases} that execute
-in the context of a @dfn{test-runner}. This specification
-also supports writing new test-runners, to allow customization
-of reporting and processing the result of running test suites.
+This defines an API for writing @dfn{test suites}, to make it easy to
+portably test Scheme APIs, libraries, applications, and implementations.
+A test suite is a collection of @dfn{test cases} that execute in the
+context of a @dfn{test-runner}. This specification also supports
+writing new test-runners, to allow customization of reporting and
+processing the result of running test suites.
@node SRFI 64 Rationale
@subsubsection SRFI 64 Rationale
-
-The Scheme community needs a standard for writing test suites.
-Every SRFI or other library should come with a test suite.
-Such a test suite must be portable, without requiring any
-non-standard features, such as modules. The test suite implementation
-or "runner" need not be portable, but it is desirable that it be
-possible to write a portable basic implementation.
+The Scheme community needs a standard for writing test suites. Every
+SRFI or other library should come with a test suite. Such a test suite
+must be portable, without requiring any non-standard features, such as
+modules. The test suite implementation or "runner" need not be
+portable, but it is desirable that it be possible to write a portable
+basic implementation.
There are other testing frameworks written in Scheme, including
-@url{https://docs.racket-lang.org/rackunit/, RackUnit}However RackUnit is not
portable.
-It is also a bit on the verbose side.
-It would be useful to have a bridge between this framework and RackUnit
-so RackUnit tests could run under this framework and vice versa.
-There exists also at least one Scheme wrapper providing a Scheme interface
-to the ``standard''@url{https://www.junit.org/, JUnit} API for Java.
-It would be useful to have a bridge so that tests written using this
-framework can run under a JUnit runner.
-Neither of these features are part of this specification.
-
-This API makes use of implicit dynamic state, including an
-implicit ``test runner''. This makes the API convenient
-and terse to use, but it may be a little less elegant and
``compositional''than using explicit test objects, such as JUnit-style
frameworks.
-It is not claimed to follow either object-oriented or functional design
-principles, but I hope it is useful and convenient to use and extend.
+@url{https://docs.racket-lang.org/rackunit/, RackUnit}. However
+RackUnit is not portable. It is also a bit on the verbose side. It
+would be useful to have a bridge between this framework and RackUnit so
+RackUnit tests could run under this framework and vice versa. There
+exists also at least one Scheme wrapper providing a Scheme interface to
+the ``standard'' @url{https://www.junit.org/, JUnit} API for Java. It
+would be useful to have a bridge so that tests written using this
+framework can run under a JUnit runner. Neither of these features are
+part of this specification.
+
+This API makes use of implicit dynamic state, including an implicit
+``test runner''. This makes the API convenient and terse to use, but it
+may be a little less elegant and ``compositional'' than using explicit
+test objects, such as JUnit-style frameworks. It is not claimed to
+follow either object-oriented or functional design principles, but I
+hope it is useful and convenient to use and extend.
This proposal allows converting a Scheme source file to a
test suite by just adding a few macros. You don't have to
write the entire file in a new form, thus you don't have to
re-indent it.
-All names defined by the API start with the prefix @code{test-}All
function-like forms are defined as syntax. They may be implemented
-as functions or macros or built-ins. The reason for specifying them as
-syntax is to allow specific tests to be skipped without evaluating
sub-expressions, or for implementations
-to add features such as printing line numbers or catching exceptions.
-@c TODO: Review (delete/relocate) the following text (if any)
-@c orphaned by splicing the `(h1 Specification)' node.
-
-
-While this is a moderately complex specification,
-you should be able to write simple test suites after just reading the
-first few sections below. More advanced functionality, such
-as writing a custom test-runner, is at the end of the specification.
+All names defined by the API start with the prefix @samp{test-}. All
+function-like forms are defined as syntax. They may be implemented as
+functions or macros or built-ins. The reason for specifying them as
+syntax is to allow specific tests to be skipped without evaluating
+sub-expressions, or for implementations to add features such as printing
+line numbers or catching exceptions.
@node SRFI 64 Writing basic test suites
@subsubsection SRFI 64 Writing basic test suites
-
-Let's start with a simple example.
-This is a complete self-contained test-suite.
+Let's start with a simple example. This is a complete self-contained
+test-suite.
@lisp
;; Initialize and give a name to a simple testsuite.
@@ -81,73 +68,59 @@
(test-end "vec-test")
@end lisp
-This testsuite could be saved in its own source file.
-Nothing else is needed:
-We do not require any top-level forms, so it is easy
-to wrap an existing program or test to this form, without adding indentation.
-It is also easy to add new tests, without having to name individual
-tests (though that is optional).
-
-Test cases are executed in the context of a @dfn{test runner},
-which is a object that accumulates and reports test results.
-This specification defines how to create and use custom test runners,
-but implementations should also provide a default test runner.
-It is suggested (but not required) that loading the above
-file in a top-level environment will cause the
-tests to be executed using an implementation-specified default test runner,
-and @code{test-end} will cause a summary to be displayed
-in an implementation-specified manner.
+This testsuite could be saved in its own source file. Nothing else is
+needed: We do not require any top-level forms, so it is easy to wrap an
+existing program or test to this form, without adding indentation. It
+is also easy to add new tests, without having to name individual tests
+(though that is optional).
+
+Test cases are executed in the context of a @dfn{test runner}, which is
+a object that accumulates and reports test results. This specification
+defines how to create and use custom test runners, but implementations
+should also provide a default test runner. It is suggested (but not
+required) that loading the above file in a top-level environment will
+cause the tests to be executed using an implementation-specified default
+test runner, and @code{test-end} will cause a summary to be displayed in
+an implementation-specified manner.
@subsubheading Simple test-cases
+Primitive test cases test that a given condition is true. They may have
+a name. The core test case form is @code{test-assert}:
-Primitive test cases test that a given condition is true.
-They may have a name.
-The core test case form is @code{test-assert}:
-
+@deffn {Scheme Syntax} test-assert [test-name] expression
-@deffn {Scheme Procedure} test-assert [test-name] expression
-@c FIXME: Check deffn category and adjust '@end deffn' location
+This evaluates the @var{expression}. The test passes if the result is
+true; if the result is false, a test failure is reported. The test also
+fails if an exception is raised, assuming the implementation has a way
+to catch exceptions. How the failure is reported depends on the test
+runner environment. The @var{test-name} is a string that names the test
+case. (Though the @var{test-name} is a string literal in the examples,
+it is an expression. It is evaluated only once.) It is used when
+reporting errors, and also when skipping tests, as described below. It
+is an error to invoke @code{test-assert}if there is no current test
+runner.
@end deffn
+The following forms may be more convenient than using @code{test-assert}
+directly:
-This evaluates the @var{expression}The test passes if the result
-is true; if the result is false, a test failure is reported.
-The test also fails if an exception is raised, assuming the implementation
-has a way to catch exceptions.
-How the failure is reported depends on the test runner environment.
-The @var{test-name} is a string that names the test case.
-(Though the @var{test-name} is a string literal in the examples,
-it is an expression. It is evaluated only once.)
-It is used when reporting errors, and also when skipping tests,
-as described below.
-It is an error to invoke @code{test-assert}if there is no current test runner.
-
-The following forms may be more convenient than
-using @code{test-assert} directly:
-
-
-@deffn {Scheme Procedure} test-eqv [test-name] expected test-expr
-@c FIXME: Check deffn category and adjust '@end deffn' location
-@end deffn
-
+@deffn {Scheme Syntax} test-eqv [test-name] expected test-expr
This is equivalent to:
@lisp
-(test-assert [@var{test-name}] (eqv? @var{expected}@var{test-expr}))
+(test-assert [@var{test-name}] (eqv? @var{expected} @var{test-expr}))
@end lisp
-Similarly @code{test-equal} and @code{test-eq}are shorthand for
@code{test-assert} combined with
-@code{equal?} or @code{eq?}, respectively:
-
-
-@deffn {Scheme Procedure} test-equal [test-name] expected test-expr
-
-@deffnx {Scheme Procedure} test-eq [test-name] expected test-expr
-@c FIXME: Check deffn category and adjust '@end deffn' location
@end deffn
+Similarly @code{test-equal} and @code{test-eq} are shorthand for
+@code{test-assert} combined with @code{equal?} or @code{eq?},
+respectively:
+
+@deffn {Scheme Syntax} test-equal [test-name] expected test-expr
+@deffnx {Scheme Syntax} test-eq [test-name] expected test-expr
Here is a simple example:
@@ -155,56 +128,50 @@
(define (mean x y) (/ (+ x y) 2.0))
(test-eqv 4 (mean 3 5))
@end lisp
+@end deffn
For testing approximate equality of inexact reals
we can use @code{test-approximate}:
+@deffn {Scheme Syntax} test-approximate [test-name] expected test-expr error
-@deffn {Scheme Procedure} test-approximate [test-name] expected test-expr error
-@c FIXME: Check deffn category and adjust '@end deffn' location
-@end deffn
-
-
-This is equivalent to (except that each argument is only evaluated once):
+This is equivalent to (except that each argument is only evaluated
+once):
@lisp
(test-assert [test-name]
(and (>= test-expr (- expected error))
(<= test-expr (+ expected error))))
@end lisp
+@end deffn
@subsubheading Tests for catching errors
+We need a way to specify that evaluation @emph{should} fail. This
+verifies that errors are detected when required.
-We need a way to specify that evaluation @emph{should} fail.
-This verifies that errors are detected when required.
+@deffn {Scheme Syntax} test-error [[test-name] error-type] test-expr
+Evaluating @var{test-expr} is expected to signal an error. The kind of
+error is indicated by @var{error-type}.
-@deffn {Scheme Procedure} test-error [[test-name] error-type] test-expr
-@c FIXME: Check deffn category and adjust '@end deffn' location
-@end deffn
-
-
-Evaluating @var{test-expr} is expected to signal an error.
-The kind of error is indicated by @var{error-type}.
-
-If the @var{error-type} is left out, or it is
-@code{#t}, it means "some kind of unspecified error should be signaled".
-For example:
+If the @var{error-type} is left out, or it is @code{#t}, it means "some
+kind of unspecified error should be signaled". For example:
@lisp
(test-error #t (vector-ref '#(1 2) 9))
@end lisp
This specification leaves it implementation-defined (or for a future
-specification) what form @var{test-error} may take,
-though all implementations must allow @code{#t}Some implementations may support
-@url{https://srfi.schemers.org/srfi-35/srfi-35.html, SRFI-35's conditions},
-but these are only standardized for
-@url{https://srfi.schemers.org/srfi-36/srfi-36.html, SRFI-36's I/O
conditions}, which are seldom useful in test suites.
-An implementation may also allow implementation-specific
-``exception types''For example Java-based implementations may allow
-the names of Java exception classes:
+specification) what form @var{test-error} may take, though all
+implementations must allow @code{#t}. Some implementations may support
+@url{https://srfi.schemers.org/srfi-35/srfi-35.html, SRFI-35's
+conditions}, but these are only standardized for
+@url{https://srfi.schemers.org/srfi-36/srfi-36.html, SRFI-36's I/O
+conditions}, which are seldom useful in test suites. An implementation
+may also allow implementation-specific ``exception types''. For example
+Java-based implementations may allow the names of Java exception
+classes:
@lisp
;; Kawa-specific example
@@ -213,31 +180,23 @@
An implementation that cannot catch exceptions should skip
@code{test-error} forms.
+@end deffn
@subsubheading Testing syntax
-
-Testing syntax is tricky, especially if we want to
-check that invalid syntax is causing an error.
-The following utility function can help:
-
+Testing syntax is tricky, especially if we want to check that invalid
+syntax is causing an error. The following utility function can help:
@deffn {Scheme Procedure} test-read-eval-string string
-@c FIXME: Check deffn category and adjust '@end deffn' location
-@end deffn
-
-
-This function parses @var{string} (using @code{read})
-and evaluates the result.
-The result of evaluation is returned from @code{test-read-eval-string}An error
is signalled if there are unread characters after the
-@code{read} is done.
-For example:
-@code{(test-read-eval-string "(+ 3 4)")}@i{evaluates to}@code{7}.
-@code{(test-read-eval-string "(+ 3 4")}@i{signals an error}.
-@code{(test-read-eval-string "(+ 3 4) ")}@i{signals an error},
-because there is extra ``junk'' (@i{i.e.} a space) after the
-list is read.
+This function parses @var{string} (using @code{read}) and evaluates the
+result. The result of evaluation is returned from
+@code{test-read-eval-string}. An error is signalled if there are unread
+characters after the @code{read} is done. For example:
+@code{(test-read-eval-string "(+ 3 4)")} @i{evaluates to} @code{7}.
+@code{(test-read-eval-string "(+ 3 4")} @i{signals an error}.
+@code{(test-read-eval-string "(+ 3 4) ")} @i{signals an error}, because
+there is extra ``junk'' (@i{i.e.} a space) after the list is read.
The @code{test-read-eval-string} used in tests:
@@ -251,101 +210,81 @@
(test-equal 5 (test-read-eval-string "(+ 1 #;(* 2 3) 4)"))
(test-equal '(x z) (test-read-string "(list 'x #;'y 'z)"))
@end lisp
+@end deffn
@subsubheading Test groups and paths
-
A @dfn{test group} is a named sequence of forms containing testcases,
-expressions, and definitions.
-Entering a group sets the @dfn{test group name}; leaving a
-group restores the previous group name.
-These are dynamic (run-time) operations, and a group has no
-other effect or identity.
-Test groups are informal groupings: they are neither
-Scheme values, nor are they syntactic forms.
+expressions, and definitions. Entering a group sets the @dfn{test group
+name}; leaving a group restores the previous group name. These are
+dynamic (run-time) operations, and a group has no other effect or
+identity. Test groups are informal groupings: they are neither Scheme
+values, nor are they syntactic forms.
@c (More formal <q>test suite</q> values are introduced below.)
-
-
A test group may contain nested inner test groups.
The @dfn{test group path} is a list of the currently-active
(entered) test group names, oldest (outermost) first.
+@deffn {Scheme Syntax} test-begin suite-name [count]
-@deffn {Scheme Procedure} test-begin suite-name [count]
-@c FIXME: Check deffn category and adjust '@end deffn' location
-@end deffn
-
+A @code{test-begin} enters a new test group. The @var{suite-name}
+becomes the current test group name, and is added to the end of the test
+group path. Portable test suites should use a string literal for
+@var{suite-name}; the effect of expressions or other kinds of literals
+is unspecified.
-A @code{test-begin} enters a new test group.
-The @var{suite-name} becomes the current test group name,
-and is added to the end of the test group path.
-Portable test suites should use a string literal for @var{suite-name};
-the effect of expressions or other kinds of literals is unspecified.
-
-@b{Rationale:} In some ways using symbols would be preferable.
+@emph{Rationale:} In some ways using symbols would be preferable.
However, we want human-readable names, and standard Scheme does not
-provide a way to include spaces or mixed-case text in
-literal symbols.
+provide a way to include spaces or mixed-case text in literal symbols.
-The optional @var{count} must match the number of
-test-cases executed by this group.
-(Nested test groups count as a single test case for this count.)
-This extra test may be useful to catch cases where a test doesn't
-get executed because of some unexpected error.
+The optional @var{count} must match the number of test-cases executed by
+this group. (Nested test groups count as a single test case for this
+count.) This extra test may be useful to catch cases where a test
+doesn't get executed because of some unexpected error.
Additionally, if there is no currently executing test runner,
one is installed in an implementation-defined manner.
-
-
-@deffn {Scheme Procedure} test-end [suite-name]
-@c FIXME: Check deffn category and adjust '@end deffn' location
@end deffn
+@deffn {Scheme Syntax} test-end [suite-name]
A @code{test-end} leaves the current test group.
An error is reported if the @var{suite-name} does not
match the current test group name.
-@c If it does match an earlier
-@c name in the test group path, intervening groups are left.
-
+@c If it does match an earlier name in the test group path, intervening
+@c groups are left.
+Additionally, if the matching @code{test-begin}installed a new
+test-runner, then the @code{test-end} will uninstall it, after reporting
+the accumulated test results in an implementation-defined manner.
+@end deffn
-Additionally, if the matching @code{test-begin}installed a new test-runner,
then the @code{test-end}will uninstall it, after reporting the accumulated test
-results in an implementation-defined manner.
-
-@lisp
-(@b{test-group}@var{suite-name}@var{decl-or-expr} @dots{})
-@end lisp
+@deffn {Scheme Syntax} test-group suite-name decl-or-expr @dots{}
Equivalent to:
@lisp
-(if (not (test-to-skip% @var{suite-name}))
+(if (not (test-to-skip% (var suite-name)))
(dynamic-wind
- (lambda () (test-begin @var{suite-name}))
- (lambda () @var{decl-or-expr} @dots{})
- (lambda () (test-end @var{suite-name}))))
+ (lambda () (test-begin (var suite-name)))
+ (lambda () (var decl-or-expr) ...)
+ (lambda () (test-end (var suite-name)))))
@end lisp
This is usually equivalent to executing the @var{decl-or-expr}s
within the named test group. However, the entire group is skipped
if it matched an active @code{test-skip} (see later).
Also, the @code{test-end} is executed in case of an exception.
+@end deffn
@subsubheading Handling set-up and cleanup
+@deffn {Scheme Syntax} test-group-with-cleanup suite-name decl-or-expr @dots{}
cleanup-form
-
-@deffn {Scheme Procedure} test-group-with-cleanup suite-name decl-or-expr
@dots{} cleanup-form
-@c FIXME: Check deffn category and adjust '@end deffn' location
-@end deffn
-
-
-Execute each of the @var{decl-or-expr} forms in order
-(as in a @var{<body>}),
-and then execute the @var{cleanup-form}The latter should be executed even if
-one of a @var{decl-or-expr} forms raises an exception
-(assuming the implementation has a way to catch exceptions).
+Execute each of the @var{decl-or-expr} forms in order (as in a
+@var{<body>}), and then execute the @var{cleanup-form}. The latter
+should be executed even if one of a @var{decl-or-expr} forms raises an
+exception (assuming the implementation has a way to catch exceptions).
For example:
@@ -355,103 +294,77 @@
(do-a-bunch-of-tests f)
(close-output-port f)))
@end lisp
-
-@b{Erratum
note:}@url{https://github.com/scheme-requests-for-implementation/srfi-64/blob/4470ffdec71b1cf61633b664958a3ce5e6997710/srfi-64.html,
Earlier versions} had a non-working example.
-
-@c <h3>Test suites</h3>
-@c <p>
-@c <i>(Not sure how useful this is, given <code>test-group</code>).</i>
-@c <p>A <dfn>test suite</dfn> is a test group that can (and must) be
-@c executed explicitly.
-@c <pre>
-@c (test-suite <var>suite-name</var> <var>decl-or-expr</var> @dots{})
-@c </pre>
-@c <p>
-@c The <df>test suite path</dfn> is the list of names of currently
-@c running testsuites, from outermost (oldest) to innermost (newest).
-@c <p>A <code>test-suite</code> form is equivalent to:
-@c <pre>
-@c (test-suite-register <var>suite-name</var>
-@c (test-group <var>suite-name</var> <var>decl-or-expr</var> @dots{}))
-@c </pre>
-@c You can run previously registered test suite:
-@c <pre>
-@c (test-suite-run <var>suite-name</var>)
-@c </pre>
-
+@end deffn
@node SRFI 64 Conditonal test-suites and other advanced features
@subsubsection SRFI 64 Conditonal test-suites and other advanced features
-
The following describes features for controlling which tests to execute,
or specifying that some tests are @emph{expected} to fail.
@subsubheading Test specifiers
-
-Sometimes we want to only run certain tests, or we know that
-certain tests are expected to fail.
-A @dfn{test specifier} is one-argument function that takes a test-runner
-and returns a boolean. The specifier may be run before a test is performed,
-and the result may control whether the test is executed.
-For convenience, a specifier may also be a non-procedure value,
-which is coerced to a specifier procedure, as described below for
-@var{count} and @var{name}.
+Sometimes we want to only run certain tests, or we know that certain
+tests are expected to fail. A @dfn{test specifier} is one-argument
+function that takes a test-runner and returns a boolean. The specifier
+may be run before a test is performed, and the result may control
+whether the test is executed. For convenience, a specifier may also be
+a non-procedure value, which is coerced to a specifier procedure, as
+described below for @var{count} and @var{name}.
A simple example is:
@lisp
-(if @var{some-condition} (test-skip 2)) ;; skip next 2 tests
+(if (var some-condition) (test-skip 2)) ;; skip next 2 tests
@end lisp
-
@deffn {Scheme Procedure} test-match-name name
-The resulting specifier matches if the current test name (asreturned
by@code{test-runner-test-name}) is@code{equal?}toname.
+The resulting specifier matches if the current test name (as returned by
+@code{test-runner-test-name}) is @code{equal?} to @var{name}.
@end deffn
+@deffn {Scheme Syntax} test-match-nth n [count]
-@deffn {Scheme Procedure} test-match-nth n [count]
-
-This evaluates to a@emph{stateful}predicate: A counter keeps track ofhow many
times it has been called.The predicate matches then'th time it is
called(where@code{1}is the first time), andthe next
-@c FIXME: Check deffn category and adjust '@end deffn' location
+This evaluates to a @emph{stateful} predicate: A counter keeps track of
+how many times it has been called. The predicate matches the @var{n}'th
+time it is called (where @code{1} is the first time), and the next
+@samp{(- @var{count} 1)} times, where @var{count} defaults to @code{1}.
@end deffn
+@deffn {Scheme Syntax} test-match-any specifier @dots{}
-@c FIXME: Check deffn category and adjust '@end deffn' location
+The resulting specifier matches if any @var{specifier} matches. Each
+@var{specifier} is applied, in order, so side-effects from a later
+@var{specifier} happen even if an earlier @var{specifier} is true.
@end deffn
+@deffn {Scheme Syntax} test-match-all specifier @dots{}
-@c FIXME: Check deffn category and adjust '@end deffn' location
+The resulting specifier matches if each @var{specifier} matches. Each
+@var{specifier} is applied, in order, so side-effects from a later
+@var{specifier} happen even if an earlier @var{specifier} is false.
@end deffn
+@var{count} @i{(i.e. an integer)}
+Convenience short-hand for: @samp{(test-match-nth 1 @var{count})}.
-@var{count}@i{(i.e. an integer)}
-Convenience short-hand for: @code{(test-match-nth 1 @var{count})}.
-
-@var{name}@i{(i.e. a string)}
-Convenience short-hand for @code{(test-match-name @var{name})}.
+@var{name} @i{(i.e. a string)}
+Convenience short-hand for @samp{(test-match-name @var{name})}.
@subsubheading Skipping selected tests
-
In some cases you may want to skip a test.
+@deffn {Scheme Syntax} test-skip specifier
-@deffn {Scheme Procedure} test-skip specifier
-@c FIXME: Check deffn category and adjust '@end deffn' location
-@end deffn
-
-
-Evaluating @code{test-skip} adds the
-resulting @var{specifier}to the set of currently active skip-specifiers.
-Before each test (or @code{test-group})
-the set of active skip-specifiers are applied to the active test-runner.
-If any specifier matches, then the test is skipped.
+Evaluating @code{test-skip} adds the resulting @var{specifier} to the
+set of currently active skip-specifiers. Before each test (or
+@code{test-group}) the set of active skip-specifiers are applied to the
+active test-runner. If any specifier matches, then the test is skipped.
-For convenience, if the @var{specifier} is a string that
-is syntactic sugar for @code{(test-match-name @var{specifier})}For example:
+For convenience, if the @var{specifier} is a string that is syntactic
+sugar for @code{(test-match-name @var{specifier})}. For example:
@lisp
(test-skip "test-b")
@@ -459,7 +372,8 @@
(test-assert "test-b") ;; skipped
@end lisp
-Any skip specifiers introduced by a @code{test-skip}are removed by a following
non-nested @code{test-end}.
+Any skip specifiers introduced by a @code{test-skip} are removed by a
+following non-nested @code{test-end}.
@lisp
(test-begin "group1")
@@ -468,21 +382,16 @@
(test-end "group1") ;; Undoes the prior test-skip
(test-assert "test-a") ;; executed
@end lisp
+@end deffn
@subsubheading Expected failures
+Sometimes you know a test case will fail, but you don't have time to or
+can't fix it. Maybe a certain feature only works on certain platforms.
+However, you want the test-case to be there to remind you to fix it.
+You want to note that such tests are expected to fail.
-Sometimes you know a test case will fail, but you don't have time
-to or can't fix it. Maybe a certain feature only works on certain platforms.
-However, you want the test-case to be there
-to remind you to fix it. You want to note that
-such tests are expected to fail.
-
-
-@deffn {Scheme Procedure} test-expect-fail specifier
-@c FIXME: Check deffn category and adjust '@end deffn' location
-@end deffn
-
+@deffn {Scheme Syntax} test-expect-fail specifier
Matching tests (where matching is defined as in @code{test-skip})
are expected to fail. This only affects test reporting,
@@ -490,102 +399,95 @@
@lisp
(test-expect-fail 2)
-(test-eqv @dots{}) ;; expected to fail
-(test-eqv @dots{}) ;; expected to fail
-(test-eqv @dots{}) ;; expected to pass
+(test-eqv ...) ;; expected to fail
+(test-eqv ...) ;; expected to fail
+(test-eqv ...) ;; expected to pass
@end lisp
+@end deffn
@node SRFI 64 Test-runner
@subsubsection SRFI 64 Test-runner
-
-A @dfn{test-runner} is an object that runs a test-suite,
-and manages the state. The test group path, and the sets skip and
-expected-fail specifiers are part of the test-runner.
-A test-runner will also typically accumulate statistics about executed tests.
-
-
+A @dfn{test-runner} is an object that runs a test-suite, and manages the
+state. The test group path, and the sets skip and expected-fail
+specifiers are part of the test-runner. A test-runner will also
+typically accumulate statistics about executed tests,
@deffn {Scheme Procedure} test-runner? value
-True iff@code{value}is a test-runner object.
+True if and only if @var{value} is a test-runner object.
@end deffn
+@deffn {Scheme Parameter} test-runner-current
+@deffnx {Scheme Parameter} test-runner-current runner
-@deffn {Scheme Procedure} test-runner-current
-
-@deffnx {Scheme Procedure} test-runner-current runner
-
-Get or set the current test-runner.If an implementation supports parameter
objects(as in@url{https://srfi.schemers.org/srfi-39/srfi-39.html,
SRFI-39}),then@code{test-runner-current}can be a parameter
object.Alternatively,@code{test-runner-current}may be implementedas a macro or
functionthat uses a fluid or thread-local variable, or a plain global variable.
+Get or set the current test-runner.
@end deffn
-
@deffn {Scheme Procedure} test-runner-get
-Same as
-@c FIXME: Check deffn category and adjust '@end deffn' location
+Same as @code{(test-runner-current)}, but throws an exception if there
+is no current test-runner.
@end deffn
-
-
@deffn {Scheme Procedure} test-runner-simple
-Creates a new simple test-runner, that prints errors and a summaryon the
standard output port.
+Creates a new simple test-runner, that prints errors and a summary on
+the standard output port.
@end deffn
-
@deffn {Scheme Procedure} test-runner-null
-Creates a new test-runner, that does nothing with the test results.This is
mainly meant for extending when writing a custom runner.
+Creates a new test-runner, that does nothing with the test results.
+This is mainly meant for extending when writing a custom runner.
@end deffn
-Implementations @emph{may} provide other test-runners, perhaps
-a @code{(test-runner-gui)}.
-
-
@deffn {Scheme Procedure} test-runner-create
-Create a new test-runner. Equivalent to
-@c FIXME: Check deffn category and adjust '@end deffn' location
+Create a new test-runner. Equivalent to @samp{((test-runner-factory))}.
@end deffn
+@deffn {Scheme Parameter} test-runner-factory
+@deffnx {Scheme Parameter} test-runner-factory factory
-
-@deffn {Scheme Procedure} test-runner-factory
-
-@deffnx {Scheme Procedure} test-runner-factory factory
-
-Get or set the current test-runner factory.A factory is a zero-argument
function that creates a new test-runner.The default value
is@code{test-runner-simple},but implementations may provide a way to override
the default.As with@code{test-runner-current}, this may be a parameter
object,or use a per-thread, fluid, or global variable.
+Get or set the current test-runner factory. A factory is a
+zero-argument function that creates a new test-runner. The default
+value is @code{test-runner-simple}.
@end deffn
@subsubheading Running specific tests with a specified runner
-
-
@deffn {Scheme Procedure} test-apply [runner] specifier @dots{} procedure
-Callsprocedurewith no arguments using the specifiedrunneras the current
test-runner.Ifrunneris omitted,then
-@c FIXME: Check deffn category and adjust '@end deffn' location
+Calls @var{procedure} with no arguments using the specified @var{runner}
+as the current test-runner. If @var{runner} is omitted, then
+@code{(test-runner-current)} is used. (If there is no current runner,
+one is created as in @code{test-begin}.) If one or more
+@var{specifier}s are listed then only tests matching the
+@var{specifier}s are executed. A @var{specifier} has the same form as
+one used for @code{test-skip}. A test is executed if it matches any of
+the @var{specifier}s in the @code{test-apply} @emph{and} does not match
+any active @code{test-skip} specifiers.
@end deffn
+@deffn {Scheme Syntax} test-with-runner runner decl-or-expr @dots{}
-@c FIXME: Check deffn category and adjust '@end deffn' location
+Executes each @var{decl-or-expr} in order in a context where the current
+test-runner is @var{runner}.
@end deffn
-
@node SRFI 64 Test results
@subsubsection SRFI 64 Test results
-
Running a test sets various status properties in the current test-runner.
This can be examined by a custom test-runner,
or (more rarely) in a test-suite.
@subsubheading Result kind
-
Running a test may yield one of the following
status symbols:
+
@table @asis
@item @code{'pass}
The test passed, as expected.
@@ -601,70 +503,76 @@
@item @code{'skip}
The test was skipped.
-
@end table
-
@deffn {Scheme Procedure} test-result-kind [runner]
-Returns one of the above result codes from the most recent
tests.Returns@code{#f}if no tests have been run yet.If we've started on a new
test, but don't have a result yet,then the result kind is@code{'xfail}if the
test is expected to fail,@code{'skip}if the test is supposed to be
skipped,or@code{#f}otherwise.
+Returns one of the above result codes from the most recent tests.
+Returns @code{#f} if no tests have been run yet. If we've started on a
+new test, but don't have a result yet, then the result kind is
+@code{'xfail} if the test is expected to fail, @code{'skip} if the test
+is supposed to be skipped, or @code{#f} otherwise.
@end deffn
-
@deffn {Scheme Procedure} test-passed? [runner]
-True if the value of
-
-@deffnx {Scheme Procedure} test-result-kind [runner]
-
-is one of@code{'pass}or@code{'xpass}This is a convenient shorthand that might
be usefulin a test suite to only run certain tests if the previous test passed.
+True if the value of @samp{(test-result-kind [@var{runner}])} is one of
+@code{'pass} or @code{'xpass}. This is a convenient shorthand that
+might be useful in a test suite to only run certain tests if the
+previous test passed.
@end deffn
@subsubheading Test result properties
+A test runner also maintains a set of more detailed
+``result@tie{}properties'' associated with the current or most recent
+test. (I.e. the properties of the most recent test are available as
+long as a new test hasn't started.) Each property has a name (a symbol)
+and a value (any value). Some properties are standard or set by the
+implementation; implementations can add more.
-A test runner also maintains a set of more detailed ``result
properties''associated with the current or most recent test. (I.e. the
properties of the
-most recent test are available as long as a new test hasn't started.)
-Each property has a name (a symbol) and a value (any value).
-Some properties are standard or set by the implementation;
-implementations can add more.
-
+@deffn {Scheme Procedure} test-result-ref runner pname [default]
-@deffn {Scheme Procedure} test-result-ref runner ' pname [default]
-
-Returns the property value associated with thepnameproperty name.If there is
no value associated with@code{'pname}returndefault,or@code{#f}ifdefaultisn't
specified.
+Returns the property value associated with the @var{pname} property name
+(a symbol). If there is no value associated with @var{pname} return
+@var{default}, or @code{#f} if @var{default} isn't specified.
@end deffn
+@deffn {Scheme Syntax} test-result-set! runner pname value
-@deffn {Scheme Procedure} test-result-set! runner ' pname value
-
-Sets the property value associated with thepnameproperty name tovalueUsually
implementation code should call this function, but it may beuseful for a custom
test-runner to add extra properties.
+Sets the property value associated with the @var{pname} property name to
+@var{value}. Usually implementation code should call this function, but
+it may be useful for a custom test-runner to add extra properties.
@end deffn
+@deffn {Scheme Procedure} test-result-remove runner pname
-@deffn {Scheme Procedure} test-result-remove runner ' pname
-
-Remove the property with the name@code{'pname}.
+Remove the property with the name @var{pname}.
@end deffn
-
@deffn {Scheme Procedure} test-result-clear runner
-Remove all result properties.The implementation automatically
calls@code{test-result-clear}at the start of a@code{test-assert}and similar
procedures.
+Remove all result properties. The implementation automatically calls
+@code{test-result-clear} at the start of a @code{test-assert} and
+similar procedures.
@end deffn
-
@deffn {Scheme Procedure} test-result-alist runner
-Returns an association list of the current result properties.It is unspecified
if the result shares state with the test-runner.The result should not be
modified; on the other hand, the resultmay be implicitly modified by
future@code{test-result-set!}or@code{test-result-remove}calls.However,
a@code{test-result-clear}does not modify the returnedalist. Thus you
can``archive''result objects from previous runs.
+Returns an association list of the current result properties. It is
+unspecified if the result shares state with the test-runner. The result
+should not be modified; on the other hand, the result may be implicitly
+modified by future @code{test-result-set!} or @code{test-result-remove}
+calls. However, a @code{test-result-clear} does not modify the returned
+alist. Thus you can ``archive'' result objects from previous runs.
@end deffn
@subsubheading Standard result properties
-
The set of available result properties is implementation-specific.
However, it is suggested that the following might be provided:
@table @asis
+
@item @code{'result-kind}
The result kind, as defined previously.
This is the only mandatory result property.
@@ -691,121 +599,113 @@
@item @code{'actual-error}
The error value, if an error was signalled and it is known.
The actual error value is implementation-defined.
-
@end table
@node SRFI 64 Writing a new test-runner
@subsubsection SRFI 64 Writing a new test-runner
-
-This section specifies how to write a test-runner.
-It can be ignored if you just want to write test-cases.
+This section specifies how to write a test-runner. It can be ignored if
+you just want to write test-cases.
@subsubheading Call-back functions
-
These call-back functions are ``methods'' (in the object-oriented sense)
-of a test-runner. A method @code{test-runner-on-@var{event}}is called by the
implementation when @var{event} happens.
+of a test-runner. A method @code{test-runner-on-@var{event}} is called
+by the implementation when @var{event} happens.
-To define (set) the callback function for @var{event} use the following
expression.
-(This is normally done when initializing a test-runner.)
+To define (set) the callback function for @var{event} use the following
+expression. (This is normally done when initializing a test-runner.)
-@code{(test-runner-on-@var{event}! @var{runner}@var{event-function})}
+@code{(test-runner-on-@var{event}! @var{runner} @var{event-function})}
-An @var{event-function} takes a test-runner argument, and possibly other
arguments, depending on the @var{event}.
+An @var{event-function} takes a test-runner argument, and possibly other
+arguments, depending on the @var{event}.
To extract (get) the callback function for @var{event} do this:
-@code{(test-runner-on-@var{event}@var{runner})}
+@code{(test-runner-on-@var{event} @var{runner})}
-To extract call the callback function for @var{event} use the following
expression.
-(This is normally done by the implementation core.)
-@code{((test-runner-on-@var{event}@var{runner}) @var{runner}@var{other-args}
@dots{})}
+To extract call the callback function for @var{event} use the following
+expression. (This is normally done by the implementation core.)
+@samp{((test-runner-on-@var{event} @var{runner}) @var{runner}
+@var{other-args} @dots{})}.
The following call-back hooks are available.
-
@deffn {Scheme Procedure} test-runner-on-test-begin runner
-
@deffnx {Scheme Procedure} test-runner-on-test-begin! runner
on-test-begin-function
+@deffnx {Scheme Procedure} on-test-begin-function runner
-@deffnx {Scheme Procedure} on-test-begin-function runner
+The @var{on-test-begin-function} is called at the start of an
+individual testcase, before the test expression (and expected value) are
+evaluated.
-Theon-test-begin-functionis called at the start of anindividual testcase,
before the test expression (and expected value) areevaluated.
@end deffn
-
@deffn {Scheme Procedure} test-runner-on-test-end runner
-
@deffnx {Scheme Procedure} test-runner-on-test-end! runner on-test-end-function
+@deffnx {Scheme Procedure} on-test-end-function runner
-@deffnx {Scheme Procedure} on-test-end-function runner
-
-Theon-test-end-functionis called at the end of anindividual testcase, when the
result of the test is available.
+The @var{on-test-end-function} is called at the end of an
+individual testcase, when the result of the test is available.
@end deffn
-
@deffn {Scheme Procedure} test-runner-on-group-begin runner
-
@deffnx {Scheme Procedure} test-runner-on-group-begin! runner
on-group-begin-function
+@deffnx {Scheme Procedure} on-group-begin-function runner suite-name count
-@deffnx {Scheme Procedure} on-group-begin-function runner suite-name count
-
-Theon-group-begin-functionis called by a@code{test-begin},including at the
start of a@code{test-group}Thesuite-nameis a Scheme string,andcountis an
integer or@code{#f}.
+The @var{on-group-begin-function} is called by a @code{test-begin},
+including at the start of a @code{test-group}. The @var{suite-name} is
+a Scheme string, and @var{count} is an integer or @code{#f}.
@end deffn
-
@deffn {Scheme Procedure} test-runner-on-group-end runner
-
@deffnx {Scheme Procedure} test-runner-on-group-end! runner
on-group-end-function
+@deffnx {Scheme Procedure} on-group-end-function runner
-@deffnx {Scheme Procedure} on-group-end-function runner
-
-Theon-group-end-functionis called by a@code{test-end},including at the end of
a@code{test-group}.
+The @var{on-group-end-function} is called by a @code{test-end},
+including at the end of a @code{test-group}.
@end deffn
-
@deffn {Scheme Procedure} test-runner-on-bad-count runner
-
@deffnx {Scheme Procedure} test-runner-on-bad-count! runner
on-bad-count-function
+@deffnx {Scheme Procedure} on-bad-count-function runner actual-count
expected-count
-@deffnx {Scheme Procedure} on-bad-count-function runner actual-count
expected-count
-
-Called from@code{test-end}(before theon-group-end-functionis called) if
anexpected-countwas specified by the matching@code{test-begin}and
theexpected-countdoes not matchtheactual-countof tests actually executed or
skipped.
+Called from @code{test-end} (before the @var{on-group-end-function} is
+called) if an @var{expected-count} was specified by the matching
+@code{test-begin} and the @var{expected-count} does not match the
+@var{actual-count} of tests actually executed or skipped.
@end deffn
-
@deffn {Scheme Procedure} test-runner-on-bad-end-name runner
-
@deffnx {Scheme Procedure} test-runner-on-bad-end-name! runner
on-bad-end-name-function
+@deffnx {Scheme Procedure} on-bad-end-name-function runner begin-name end-name
-@deffnx {Scheme Procedure} on-bad-end-name-function runner begin-name end-name
-
-Called from@code{test-end}(before theon-group-end-functionis called) if
asuite-namewas specified, and it did not that thename in the
matching@code{test-begin}.
+Called from @code{test-end} (before the @var{on-group-end-function} is
+called) if a @var{suite-name} was specified, and it did not that the
+name in the matching @code{test-begin}.
@end deffn
-
@deffn {Scheme Procedure} test-runner-on-final runner
-
@deffnx {Scheme Procedure} test-runner-on-final! runner on-final-function
+@deffnx {Scheme Procedure} on-final-function runner
-@deffnx {Scheme Procedure} on-final-function runner
-
-Theon-final-functiontakes one parameter (a test-runner)and typically displays
a summary (count) of the tests.Theon-final-functionis called after called
theon-group-end-functioncorrespondiong to the outermost@code{test-end}The
default value is@code{test-on-final-simple}which writesto the standard output
port the number of tests of the various kinds.
+The @var{on-final-function} takes one parameter (a test-runner) and
+typically displays a summary (count) of the tests. The
+@var{on-final-function} is called after called the
+@var{on-group-end-function} correspondiong to the outermost
+@code{test-end}. The default value is @code{test-on-final-simple} which
+writes to the standard output port the number of tests of the various
+kinds.
@end deffn
-
-The default test-runner returned by@code{test-runner-simple}uses the following
call-back functions:
+The default test-runner returned by @code{test-runner-simple} uses the
+following call-back functions:
@deffn {Scheme Procedure} test-on-test-begin-simple runner
-
@deffnx {Scheme Procedure} test-on-test-end-simple runner
-
@deffnx {Scheme Procedure} test-on-group-begin-simple runner suite-name count
-
@deffnx {Scheme Procedure} test-on-group-end-simple runner
-
@deffnx {Scheme Procedure} test-on-bad-count-simple runner actual-count
expected-count
-
@deffnx {Scheme Procedure} test-on-bad-end-name-simple runner begin-name
end-name
You can call those if you want to write your own test-runner.
@@ -813,76 +713,71 @@
@subsubheading Test-runner components
-
-The following functions are for accessing the other components of a
test-runner.
-They would normally only be used to write a new test-runner or
-a match-predicate.
-
+The following functions are for accessing the other components of a
+test-runner. They would normally only be used to write a new
+test-runner or a match-predicate.
@deffn {Scheme Procedure} test-runner-pass-count runner
Returns the number of tests that passed, and were expected to pass.
@end deffn
-
@deffn {Scheme Procedure} test-runner-fail-count runner
Returns the number of tests that failed, but were expected to pass.
@end deffn
-
@deffn {Scheme Procedure} test-runner-xpass-count runner
Returns the number of tests that passed, but were expected to fail.
@end deffn
-
@deffn {Scheme Procedure} test-runner-xfail-count runner
Returns the number of tests that failed, and were expected to pass.
@end deffn
-
@deffn {Scheme Procedure} test-runner-skip-count runner
Returns the number of tests or test groups that were skipped.
@end deffn
-
@deffn {Scheme Procedure} test-runner-test-name runner
-Returns the name of the current test or test group, as a string.During
execution of@code{test-begin}this is the name of thetest group; during the
execution of an actual test, this is the nameof the test-case.If no name was
specified, the name is the empty string.
+Returns the name of the current test or test group, as a string. During
+execution of @code{test-begin} this is the name of the test group;
+during the execution of an actual test, this is the name of the
+test-case. If no name was specified, the name is the empty string.
@end deffn
-
@deffn {Scheme Procedure} test-runner-group-path runner
-A list of names of groups we're nested in, with the outermost group first.
+A list of names of groups we're nested in, with the outermost group
+first.
@end deffn
-
@deffn {Scheme Procedure} test-runner-group-stack runner
-A list of names of groups we're nested in, with the outermost group last.(This
is more efficient than@code{test-runner-group-path},since it doesn't require
any copying.)
+A list of names of groups we're nested in, with the outermost group
+last. (This is more efficient than @code{test-runner-group-path}, since
+it doesn't require any copying.)
@end deffn
-
@deffn {Scheme Procedure} test-runner-aux-value runner
-
@deffnx {Scheme Procedure} test-runner-aux-value! runner on-test
-Get or set the@code{aux-value}field of a test-runner.This field is not used by
this API or the@code{test-runner-simple}test-runner, but may be used by custom
test-runners to store extra state.
+Get or set the @code{aux-value} field of a test-runner. This field is
+not used by this API or the @code{test-runner-simple} test-runner, but
+may be used by custom test-runners to store extra state.
@end deffn
-
@deffn {Scheme Procedure} test-runner-reset runner
-Resets the state of therunnerto its initial state.
+Resets the state of the @var{runner} to its initial state.
@end deffn
@subsubheading Example
-
This is an example of a simple custom test-runner.
Loading this program before running a test-suite will install
it as the default test runner.
Diff finished. Wed Oct 2 15:59:15 2024
Looking at it more closely, it looks pretty much left alone. It's only been re-flowed and some typos or extraneous comments fixed (I had submitted a bunch of typo fixes to upstream but it seems I had missed to send some). I'm also attaching the two versions (original snarfed vs submitted) for reference. [0] To snarf it on your own, you can use the tool I had shared here: https://lists.gnu.org/r/guile-devel/2023-12/msg00067.html
srfi-64-resnarfed.texi
Description: original
srfi-64-documented.texi
Description: documented
-- Thanks, Maxim
