Speccing Test.pm?
Howdy, The test suite is considered "official" as in "everything that passes the (completed) test suite may name itself Perl 6", and nearly all of these files 'use Test'; However we don't ship an "official" Test.pm, nor do we define which test functions it should contain and export by default, nor their semantics. Now this may sound a bit theoretical and far-fetched, but we've actually encountered test files that contain tests which are only in Rakudo's Test.pm (probably my fault), and otoh there are a few functions in pugs' Test.pm that are not used (for example unlike(), which is only used in t/02-test-pm/1-basic.t to test unlike()). So how should we proceed? Should I assemble a list of commonly used test functions and remove all others both in the Test.pm's and the test files? And then? Spec it? Or ship a prototype Test.pm as "official"? Cheers, Moritz -- Moritz Lenz http://perlgeek.de/ | http://perl-6.de/
Re: Speccing Test.pm?
On Tue, Sep 02, 2008 at 02:10:39PM +0200, Moritz Lenz wrote: > The test suite is considered "official" as in "everything that passes > the (completed) test suite may name itself Perl 6", and nearly all of > these files 'use Test'; However we don't ship an "official" Test.pm, nor > do we define which test functions it should contain and export by > default, nor their semantics. > > Now this may sound a bit theoretical and far-fetched, but we've actually > encountered test files that contain tests which are only in Rakudo's > Test.pm (probably my fault), and otoh there are a few functions in pugs' > Test.pm that are not used (for example unlike(), which is only used in > t/02-test-pm/1-basic.t to test unlike()). > > So how should we proceed? Should I assemble a list of commonly used test > functions and remove all others both in the Test.pm's and the test files? I'd like to see us spec the list of test functions needed by the official test suite. If possible, I'd also like those functions to be kept on simple side, so that an implementation doesn't have to have a nearly complete implementation of Perl 6 in order to start using the suite. For example, we shouldn't require advanced typing or multimethod dispatch semantics in order for Test.pm to work. > And then? Spec it? Or ship a prototype Test.pm as "official"? I think it's good to have a prototype Test.pm that we can point to as a reference, but I don't think we need to try to designate it as being "official". Pm
Re: Speccing Test.pm?
Patrick R. Michaud wrote: On Tue, Sep 02, 2008 at 02:10:39PM +0200, Moritz Lenz wrote: And then? Spec it? Or ship a prototype Test.pm as "official"? I think it's good to have a prototype Test.pm that we can point to as a reference, but I don't think we need to try to designate it as being "official". I think we *should* have an official Test.pm, but that it should be conceptualized in a slightly different way. Or actually I have 2 alternate proposals: 1. Rather than being a "language extension", Test.pm should formally be part of the Perl 6 test suite itself, as the result of a refactoring; basically all tests have some common implementation parts and those have been refactored into a module, which is Test.pm. So then Test.pm is an integral part of the test suite itself, and if there is just one official test suite, then there is just an official Test.pm. Though for convenience this Test.pm would be installable so that other language extensions could use it in their tests, though. 2. The Perl 6 language spec itself would specify a basic set of test routines built-in to the language, in a Test namespace, much as it defines collections of routines now for such as numbers and arrays and standard I/O. And so the basic test routines would be formally defined in a Synopsis document. This shouldn't be onerous as there'd probably be only about a dozen such routines and given our experience it is probably reasonable they won't need to change much over time. So how these Test namespace routines are provided can then vary by the implementation but they would be standard to users of the language. Now a common factor to both of my proposals is that this Test.pm is intentionally kept as simple as possible and contains just the functionality needed to bootstrap the official Perl 6 test suite; if the official test suite doesn't use certain features, then they don't exist in this Test.pm, in general. There would still be room for third party Test modules, as those would be richer and provide functionality that would be useful for testing language extensions / CPAN modules but that aren't needed by the tests for Perl 6 itself. I should mention that I'm currently pursuing the #2 approach in my Muldis D language design, where the language spec itself includes a small number of common routines implementing the TAP protocol over standard I/O, so that tests written in the language can just use it without having to explicitly load a Test library like a user-defined library. An advantage of #2 is that you don't have to think so much about what other language features your Test.pm uses, since from the users' point of view they are themselves fundamental features; so implementers could even bootstrap them using native equivalents or a prelude. I also don't see the possibility of our "getting it wrong" in the design to be such a big deal, since the odds are anything we think of now will work well for many years, as Test.pm/Test::More has been fairly stable already and meanwhile Perl 6 is versioned now, so we could make an incompatible change to the Test related language spec in the future, and as long as users say "use Perl-6.0.0" their code relying on the older/current Test.pm like interface won't break. -- Darren Duncan
Re: Speccing Test.pm?
On Tue, Sep 02, 2008 at 12:32:49PM -0700, Darren Duncan wrote: > Patrick R. Michaud wrote: >> I think it's good to have a prototype Test.pm that we can point to as >> a reference, but I don't think we need to try to designate it as being >> "official". > > [...] > 2. The Perl 6 language spec itself would specify a basic set of test > routines built-in to the language, in a Test namespace, much as it > defines collections of routines now for such as numbers and arrays and > standard I/O. And so the basic test routines would be formally defined > in a Synopsis document. I disagree. The testing we're likely want to do as part of the language test suite may be substantially different from what we want to provide to module writers for testing. In particular, I think that the test suite harness should require only a minimal Perl 6 implementation (note I said "harness", not the tests themselves), whereas it's much more reasonable that a testing system used by module writers could/should assume a fully working Perl 6 implementation. It's a difference of "bootstrapping" versus "running environment". > I also don't see the possibility of our "getting it wrong" in the design > to be such a big deal, since the odds are anything we think of now will > work well for many years, as Test.pm/Test::More has been fairly stable > already and meanwhile Perl 6 is versioned now, so we could make an > incompatible change to the Test related language spec in the future, and > as long as users say "use Perl-6.0.0" their code relying on the > older/current Test.pm like interface won't break. "Perl 6 is versioned now" is a misnomer. The *spec* calls for a versioned Perl 6, but I'm not aware that any of the implementations do much with that. At any rate, relying on handling multiple versions of Perl 6 to run Test.pm is exactly one of those things I'd like to avoid in the official test suite. Pm
Re: Speccing Test.pm?
> 2. The Perl 6 language spec itself would specify a basic set of test > routines built-in to the language, in a Test namespace That sounds like a good idea, but it would require that the above Test functionality be included in the automated tests... which runs the risk of infinite recursion.
Re: What happened to "err" operator?
> On Sun, Aug 31, 2008 at 04:28:36PM -0500, John M. Dlugosz wrote: >> Has the "err" operator, as a low-precidence version of //, been removed? > > Yes. > It could be recycled as a "fuzzy Boolean", returning a fractional value between +1 and -1, indicating the confidence with which the result is offerred. (As in "err, I'm not sure". :-)* ) -- Email and shopping with the feelgood factor! 55% of income to good causes. http://www.ippimail.com
Re: What happened to "err" operator?
I think you're thinking of the "erm" operator... But back to "orelse" - is the only difference between "and"/"or" and "andthen"/"orelse" the fact that the result of the lhs gets passed as a parameter into the rhs? 'Cause I don't see the difference between "short circuit" and "proceed on success/failure". On 9/2/08, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: >> On Sun, Aug 31, 2008 at 04:28:36PM -0500, John M. Dlugosz wrote: >>> Has the "err" operator, as a low-precidence version of //, been removed? >> >> Yes. >> > It could be recycled as a "fuzzy Boolean", returning a fractional value > between +1 and -1, indicating the confidence with which the result is > offerred. (As in "err, I'm not sure". :-)* ) > > > -- > > Email and shopping with the feelgood factor! > 55% of income to good causes. http://www.ippimail.com > > -- Sent from Gmail for mobile | mobile.google.com Mark J. Reed <[EMAIL PROTECTED]>
Re: What happened to "err" operator?
On Tue, Sep 02, 2008 at 07:56:33PM -0400, Mark J. Reed wrote:
: I think you're thinking of the "erm" operator...
:
: But back to "orelse" - is the only difference between "and"/"or" and
: "andthen"/"orelse" the fact that the result of the lhs gets passed as
: a parameter into the rhs? 'Cause I don't see the difference between
: "short circuit" and "proceed on success/failure".
They're also different in the sense that "and"/"or" are boolean tests
while "andthen"/"orelse" are definedness tests, as currently specced.
For logic programming purposes, though, it's possible that we'll end
up with something in between, because in addition to distinguishing
success from failure, we may wish also to distinguish expected failure
(normal regex backtracking, for example) from unexpected failure
(abject parser failure, for example). In other words, when you say
a() orelse b()
you might want to:
succeed on a()
trap mild failure of a() and try to succeed on b() instead
fail completely on drastic failure of a()
At the moment this three-way distinction depends on whether a() returns
defined/undefined or throws an exception. Maybe that's good enough.
I don't want to overcomplexify things, but I don't want to
undercomplexify them either. :)
In the limit, one might want the entire power of a CATCH block on
the right side of "orelse". Currently you'd have to write:
a() orelse do given $! {
when MajorMalfunction { die "oopsie" }
when * { b() }
}
or some such. Nest a few of those and it makes more sense to say
a() orelse
b() orelse
c() orelse
d() orelse
e();
CATCH {
when MajorMalfunction { die "oopsie" }
}
assuming that exceptions somehow get returned or thrown appropriately.
I suppose orelse could itself throw any exception that it doesn't want
to treat as benign, and the CATCH block would handle it. And the
definition of "benign" could then be pragmatically controlled if the
default wasn't good enough, I suppose.
Larry
Re: Speccing Test.pm?
On Tue, 2008-09-02 at 12:32 -0700, Darren Duncan wrote: > Now a common factor to both of my proposals is that this Test.pm is > intentionally kept as simple as possible and contains just the > functionality needed to bootstrap the official Perl 6 test suite; if the > official test suite doesn't use certain features, then they don't exist in > this Test.pm, in general. This doesn't quite address one assumed detail -- should the official test suite be modified to use as few (and as simple) Test.pm features as possible, so that Test.pm can then be made even simpler? This would likely make the test suite slightly clumsier in places, while making it easier for a new implementation to get enough functionality in place so that Test.pm is fully supported. > There would still be room for third party Test modules, as those would be > richer and provide functionality that would be useful for testing language > extensions / CPAN modules but that aren't needed by the tests for Perl 6 > itself. If the test suite is modified as above, then there pretty much HAVE to be additional Test modules -- people programming third-party code would go insane using only the anemic Test.pm that would be sufficient for a simplified test suite. Of course, that doesn't mean that a more extensive Test module can't be standardized, or even an official version written that all perl6's can ship with. It doesn't have to be all-encompassing, but a core set of "best practices" test tools, perhaps just taken from Perl 5 experience of the TDD folks and modified for Perl 6 differences, would be nice to rely on everywhere. -'f
