Re: [PATCH] Forking tests with Test::More
Without saying you shouldn't be sending them here, but as an aside...
Who told you to send patches to the list?
Is there a reason that the rt.cpan.org queue is no longer useful?
Adam K
Tassilo von Parseval wrote:
Hi,
I was told that Test::More patches should now go to this list so here we
go.
The attached patch serves as a draft for enabling test-scripts that fork
without the test-counter getting confused. It does so by using a
Storable imaged shared between the processes. The patch however does
need some modification because there's a race condition in there. It
uses lock_nstore and lock_retrieve to store the current test-metrics
thusly:
+sub _inc_testcount {
+my $self = shift;
+
+if( not $self->{Forked} ) {
+ lock $self->{Curr_Test};
+ $self->{Curr_Test}++;
+ return;
+}
+
+# we are running in forked mode, therefore
+# get data from disk, modify and write back
+
+my $stats = lock_retrieve( $self->{Forked} );
+$self->{Curr_Test} = ++$stats->{Curr_Test};
+$self->{Test_Results} = $stats->{Test_Results};
+lock_nstore( $stats => $self->{Forked} );
+}
This is not quite correct. Instead, the member $self->{Forked} should be
turned into a rw-filehandle to the storable image (it is the path to the
image right now) and _inc_testcount() would become something like that:
...
# we are running in forked mode, therefore
# get data from disk, modify and write back
# enter criticial region:
lock $self->{Forked}, LOCK_EX;
my $stats = fd_retrieve($self->{Forked});
$self->{Curr_Test} = ++$stats->{Curr_Test};
$self->{Test_Results} = $stats->{Test_Results};
nstore_fd( $stats => $self->{Forked} );
lock $self->{Forked}, LOCK_UN;
# criticial region left
A similar approach is needed for _store() and essentially for everything
that now uses lock_nstore/lock_retrieve.
Also, a test-case for this feature is tricky to conceive as
Test::Builder::Tester can't be used here. I supplied one but it's quite
messy.
I am right now in the middle of relocating to NY so I don't have the
time to do these modifications myself so maybe someone with more time on
his hands could look after that. It's not so tricky and mostly involves
some local changes to the enclosed patch.
Cheers,
Tassilo
Perl 6 on Solaris 10 anyone ?
Hi, Any experiments with Perl6/Parrot and Solaris Express or Solaris 10 ? Im curious how much the language has changed and how well the new VM would work in Solaris ... Is there any Solaris x86 or sparc package of the new Parrot VM ? thanks,
Re: [PATCH] Forking tests with Test::More
A far simpler solution (that I've posted before recently) is to
output test "numbers" like
.1.1
.1.2
.1.3
.2.1
.2.2
.1.4
etc where the first number signifies the thread/process and the second
is just an increasing sequence within that thread. The . is there at
the start so that Test::Harness doesn't get upset.
Interprocess comms using Storable seems like overkill and sounds like
the sort of thing that would have "fun" bugs,
F
On 3/28/06, Tassilo von Parseval <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I was told that Test::More patches should now go to this list so here we
> go.
>
> The attached patch serves as a draft for enabling test-scripts that fork
> without the test-counter getting confused. It does so by using a
> Storable imaged shared between the processes. The patch however does
> need some modification because there's a race condition in there. It
> uses lock_nstore and lock_retrieve to store the current test-metrics
> thusly:
>
> +sub _inc_testcount {
> +my $self = shift;
> +
> +if( not $self->{Forked} ) {
> + lock $self->{Curr_Test};
> + $self->{Curr_Test}++;
> + return;
> +}
> +
> +# we are running in forked mode, therefore
> +# get data from disk, modify and write back
> +
> +my $stats = lock_retrieve( $self->{Forked} );
> +$self->{Curr_Test} = ++$stats->{Curr_Test};
> +$self->{Test_Results} = $stats->{Test_Results};
> +lock_nstore( $stats => $self->{Forked} );
> +}
>
> This is not quite correct. Instead, the member $self->{Forked} should be
> turned into a rw-filehandle to the storable image (it is the path to the
> image right now) and _inc_testcount() would become something like that:
>
> ...
> # we are running in forked mode, therefore
> # get data from disk, modify and write back
>
> # enter criticial region:
>
> lock $self->{Forked}, LOCK_EX;
> my $stats = fd_retrieve($self->{Forked});
> $self->{Curr_Test} = ++$stats->{Curr_Test};
> $self->{Test_Results} = $stats->{Test_Results};
> nstore_fd( $stats => $self->{Forked} );
> lock $self->{Forked}, LOCK_UN;
>
> # criticial region left
>
> A similar approach is needed for _store() and essentially for everything
> that now uses lock_nstore/lock_retrieve.
>
> Also, a test-case for this feature is tricky to conceive as
> Test::Builder::Tester can't be used here. I supplied one but it's quite
> messy.
>
> I am right now in the middle of relocating to NY so I don't have the
> time to do these modifications myself so maybe someone with more time on
> his hands could look after that. It's not so tricky and mostly involves
> some local changes to the enclosed patch.
>
> Cheers,
> Tassilo
> --
> use bigint;
> $n=71423350343770280161397026330337371139054411854220053437565440;
> $m=-8,;;$_=$n&(0xff)<<$m,,$_>>=$m,,print+chr,,while(($m+=8)<=200);
>
>
>
Re: [PATCH] Forking tests with Test::More
On Tue, Mar 28, 2006 at 06:29:43PM +1000 Adam Kennedy wrote: > Without saying you shouldn't be sending them here, but as an aside... > > Who told you to send patches to the list? Andy Lester told Stas Bekman who told me. > Is there a reason that the rt.cpan.org queue is no longer useful? I don't know the reason but it certainly no longer seems useful. The queue for Test-Simple is growing continuisly. I sent this patch to rt.cpan.org six weeks ago and never heard back. Tassilo -- use bigint; $n=71423350343770280161397026330337371139054411854220053437565440; $m=-8,;;$_=$n&(0xff)<<$m,,$_>>=$m,,print+chr,,while(($m+=8)<=200);
Re: [PATCH] Forking tests with Test::More
On Tue, Mar 28, 2006 at 09:47:54AM +0100 Fergal Daly wrote: > A far simpler solution (that I've posted before recently) is to > output test "numbers" like > > .1.1 > .1.2 > .1.3 > .2.1 > .2.2 > .1.4 > > etc where the first number signifies the thread/process and the second > is just an increasing sequence within that thread. The . is there at > the start so that Test::Harness doesn't get upset. > > Interprocess comms using Storable seems like overkill and sounds like > the sort of thing that would have "fun" bugs, I really don't care how it is done, as long as it is eventually done at all. :-) As I can see it, there now exist at least two propositions on how to fix this problem. The ones responsible for Test::More/Test::Harness should take any of these proposed solutions and put them in. I just would like to be able to write test-scripts that fork without these annoying and ugly counter-mismatch messages. For that I sent one possible solution which ends my responsibilities in this matter. :-) Cheers, Tassilo -- use bigint; $n=71423350343770280161397026330337371139054411854220053437565440; $m=-8,;;$_=$n&(0xff)<<$m,,$_>>=$m,,print+chr,,while(($m+=8)<=200);
Re: [PATCH] Forking tests with Test::More
Tassilo von Parseval wrote: On Tue, Mar 28, 2006 at 09:47:54AM +0100 Fergal Daly wrote: A far simpler solution (that I've posted before recently) is to output test "numbers" like .1.1 .1.2 .1.3 .2.1 .2.2 .1.4 etc where the first number signifies the thread/process and the second is just an increasing sequence within that thread. The . is there at the start so that Test::Harness doesn't get upset. Interprocess comms using Storable seems like overkill and sounds like the sort of thing that would have "fun" bugs, I really don't care how it is done, as long as it is eventually done at all. :-) As I can see it, there now exist at least two propositions on how to fix this problem. The ones responsible for Test::More/Test::Harness should take any of these proposed solutions and put them in. I just would like to be able to write test-scripts that fork without these annoying and ugly counter-mismatch messages. For that I sent one possible solution which ends my responsibilities in this matter. :-) Cheers, Tassilo Well three, if you include my "redirect fork output to seperate files, and then merge back in at SIGCHLD/END-time" proposal, which would also allow things like testing using other languages. Really, I just want a solution that works on all platforms, and doesn't involve changing the TAP protocol, because of the number things generating TAP that aren't in Perl, but being read by Perl. Changing protocols can have big consequences. Adam K
Re: [PATCH] Forking tests with Test::More
On 3/28/06, Adam Kennedy <[EMAIL PROTECTED]> wrote: > Tassilo von Parseval wrote: > > On Tue, Mar 28, 2006 at 09:47:54AM +0100 Fergal Daly wrote: > > > >>A far simpler solution (that I've posted before recently) is to > >>output test "numbers" like > >> > >>.1.1 > >>.1.2 > >>.1.3 > >>.2.1 > >>.2.2 > >>.1.4 > >> > >>etc where the first number signifies the thread/process and the second > >>is just an increasing sequence within that thread. The . is there at > >>the start so that Test::Harness doesn't get upset. > >> > >>Interprocess comms using Storable seems like overkill and sounds like > >>the sort of thing that would have "fun" bugs, > > > > > > I really don't care how it is done, as long as it is eventually done at > > all. :-) > > > > As I can see it, there now exist at least two propositions on how to > > fix this problem. The ones responsible for Test::More/Test::Harness > > should take any of these proposed solutions and put them in. > > > > I just would like to be able to write test-scripts that fork without > > these annoying and ugly counter-mismatch messages. For that I sent one > > possible solution which ends my responsibilities in this matter. :-) > > > > Cheers, > > Tassilo > > Well three, if you include my "redirect fork output to seperate files, > and then merge back in at SIGCHLD/END-time" proposal, which would also > allow things like testing using other languages. > > Really, I just want a solution that works on all platforms, and doesn't > involve changing the TAP protocol, because of the number things > generating TAP that aren't in Perl, but being read by Perl. > > Changing protocols can have big consequences. What's changing the protocol? F
Re: [PATCH] Forking tests with Test::More
What's changing the protocol? As I understand it from http://search.cpan.org/~petdance/Test-Harness-2.56/lib/Test/Harness/TAP.pod the test number must be a number. It does refer specifically to it having to start with a digit, but I'm assuming that by number it means [1-9]\d* .1.2 would on the face of it seem to not match this pattern. Adam K
Re: [PATCH] Forking tests with Test::More
Tassilo von Parseval wrote: Hi, I was told that Test::More patches should now go to this list so here we go. The attached patch serves as a draft for enabling test-scripts that fork without the test-counter getting confused. It does so by using a Storable imaged shared between the processes. The patch however does need some modification because there's a race condition in there. It uses lock_nstore and lock_retrieve to store the current test-metrics thusly: To get back to the subject at hand, can you explain what the implications of this might be for portability, given the universality of Test::More. Wouldn't this patch introduce IPC and forking portability issues to Test::More, when it currently does have these issues (that I'm aware of). Do you think this might work better, or could be implemented as, a seperate Test::Fork type module? Adam K
Re: [PATCH] Forking tests with Test::More
That's why I said you prefix with a ".". This has the effect of making it not a number as far as TAP is concernted, instead it becomes part of the name. Of course it would be better to allow "."s in the number, that way you can check that they are increasing properly and allows you to have sub-plans (so each fork/thread/block can have it's own plan) rather than just having 1 single overall plan for the whole thing. Having 1 overall plan means that if one fork skips a test and the other does an extra test you won't notice. Given that x.x.x.x currently causes an error for TAP, changing the protocol to allow it would not break anything. But as I said, I wasn't proposing that - although I would be happy to see it, F On 3/28/06, Adam Kennedy <[EMAIL PROTECTED]> wrote: > > > What's changing the protocol? > > As I understand it from > http://search.cpan.org/~petdance/Test-Harness-2.56/lib/Test/Harness/TAP.pod > the test number must be a number. > > It does refer specifically to it having to start with a digit, but I'm > assuming that by number it means [1-9]\d* > > .1.2 would on the face of it seem to not match this pattern. > > Adam K >
Re: [PATCH] Forking tests with Test::More
On Tue, Mar 28, 2006 at 11:27:15AM +0100 Fergal Daly wrote: > That's why I said you prefix with a ".". > > This has the effect of making it not a number as far as TAP is > concernted, instead it becomes part of the name. > > Of course it would be better to allow "."s in the number, that way you > can check that they are increasing properly and allows you to have > sub-plans (so each fork/thread/block can have it's own plan) rather > than just having 1 single overall plan for the whole thing. Having 1 > overall plan means that if one fork skips a test and the other does an > extra test you won't notice. > > Given that x.x.x.x currently causes an error for TAP, changing the > protocol to allow it would not break anything. But you see, it does make a change in (or an addition to) the protocol necessary afterall since it currently doesn't work as you said yourself. No matter what you do, you either have to change Test::Harness or the module generating the TAP output. Or even both as in your case. I think any conceivable solution will have its ugly points. It's now a matter of finding a simple and robust approach and put it in. Cheers, Tassilo -- use bigint; $n=71423350343770280161397026330337371139054411854220053437565440; $m=-8,;;$_=$n&(0xff)<<$m,,$_>>=$m,,print+chr,,while(($m+=8)<=200);
Re: [PATCH] Forking tests with Test::More
On Tue, Mar 28, 2006 at 08:22:25PM +1000 Adam Kennedy wrote:
> Tassilo von Parseval wrote:
> >Hi,
> >
> >I was told that Test::More patches should now go to this list so here we
> >go.
> >
> >The attached patch serves as a draft for enabling test-scripts that fork
> >without the test-counter getting confused. It does so by using a
> >Storable imaged shared between the processes. The patch however does
> >need some modification because there's a race condition in there. It
> >uses lock_nstore and lock_retrieve to store the current test-metrics
> >thusly:
>
> To get back to the subject at hand, can you explain what the
> implications of this might be for portability, given the universality of
> Test::More.
>
> Wouldn't this patch introduce IPC and forking portability issues to
> Test::More, when it currently does have these issues (that I'm aware of).
Forking issues no, IPC issues yes. My patch doesn't make Test::More
fork, it only adds means of synchronizing processes that were forked in
a test-script.
The only unportable part is the locking of the Storable image. Ideally
this should get factored out in two methods acquire_lock() and
release_lock(), thusly:
sub acquire_lock {
my $self = shift;
if (HAVE_FLOCK) {
flock $self->{Forked} => LOCK_EX;
} elsif (HAVE_EXCL_SYSOPEN) {
select undef, undef, undef, 0.05
while not sysopen my $lockf, $self->{Lockfile}, O_CREAT|O_EXCL;
} elsif (HAVE_WHATEVER_OTHER_METHOD) {
...
} ...
}
sub release_lock {
my $self = shift;
if (HAVE_FLOCK) {
flock $self->{Forked} => LOCK_UN;
} elsif (HAVE_EXCL_SYSOPEN) {
unlink $self->{Lockfile};
} ...
}
> Do you think this might work better, or could be implemented as, a
> seperate Test::Fork type module?
It certainly could be done. But it would essentially share 90% of its
code with Test::Builder. It's simple really: Either my proposed method
is robust in which case it can go into Test-Simple. Or it isn't. Then
there's no need to implement it as a separate module. :-)
Cheers,
Tassilo
--
use bigint;
$n=71423350343770280161397026330337371139054411854220053437565440;
$m=-8,;;$_=$n&(0xff)<<$m,,$_>>=$m,,print+chr,,while(($m+=8)<=200);
Re: [PATCH] Forking tests with Test::More
On 3/28/06, Tassilo von Parseval <[EMAIL PROTECTED]> wrote: > On Tue, Mar 28, 2006 at 11:27:15AM +0100 Fergal Daly wrote: > > That's why I said you prefix with a ".". > > > > This has the effect of making it not a number as far as TAP is > > concernted, instead it becomes part of the name. > > > > Of course it would be better to allow "."s in the number, that way you > > can check that they are increasing properly and allows you to have > > sub-plans (so each fork/thread/block can have it's own plan) rather > > than just having 1 single overall plan for the whole thing. Having 1 > > overall plan means that if one fork skips a test and the other does an > > extra test you won't notice. > > > > Given that x.x.x.x currently causes an error for TAP, changing the > > protocol to allow it would not break anything. > > But you see, it does make a change in (or an addition to) the protocol > necessary afterall since it currently doesn't work as you said yourself. No change is required in TAP to suport "numbers" that begin with a "." because TAP interprets them as names not numbers. (TAP protocol says that test numbers are optional). Forget my comments about altering TAP, that was wishlist stuff and should not be confused with my suggestion for the current problem. > No matter what you do, you either have to change Test::Harness or the > module generating the TAP output. Or even both as in your case. No. > I think > any conceivable solution will have its ugly points. It's now a matter of > finding a simple and robust approach and put it in. Anything that attempts to synchronise across processes is harder to make robust and less likely to be simple. F > > Cheers, > Tassilo > -- > use bigint; > $n=71423350343770280161397026330337371139054411854220053437565440; > $m=-8,;;$_=$n&(0xff)<<$m,,$_>>=$m,,print+chr,,while(($m+=8)<=200); > >
Re: [PATCH] Forking tests with Test::More
On Mar 28, 2006, at 4:55 AM, Tassilo von Parseval wrote:
Do you think this might work better, or could be implemented as, a
seperate Test::Fork type module?
It certainly could be done. But it would essentially share 90% of its
code with Test::Builder.
Umm, subclassing? :-)
It's simple really: Either my proposed method
is robust in which case it can go into Test-Simple. Or it isn't. Then
there's no need to implement it as a separate module. :-)
I don't think it is that simple. *IF* there are any bugs in this
niche feature, then it could break the most popular test module on
CPAN. Clearly from your proposed patch, any code that needs the fork
feature of Test::More says so explicitly via the $self->{Forked}
property. So specifying a subclass instead, like a hypothetical
Test::More::Forked, should be trivial. I have not yet seen any
justification for adding this feature to Test::More itself.
Chris
--
Chris Dolan, Software Developer, Clotho Advanced Media Inc.
608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Clotho Advanced Media, Inc. - Creators of MediaLandscape Software
(http://www.media-landscape.com/) and partners in the revolutionary
Croquet project (http://www.opencroquet.org/)
[OT] TDD only works for simple things...
hi all :) for those interested in both php and perl, it seems that php's native .phpt testing feature will soon produce TAP compliant output - see greg beaver's comments here http://shiflett.org/archive/218#comments so, TAP is slowly dominating the world... but we all knew that already :) what actually prompted me to write is a comment embedded there: "Only the simplest of designs benefits from pre-coded tests, unless you have unlimited developer time." needless to say I just don't believe this. but as I try to broach the test-driven development topic with folks I hear this lots - not just that they don't have the time to use tdd, but that it doesn't work anyway for most "real" applications (where their app is sufficiently "real" or "large" or "complex" or whatever). since I'm preaching to the choir here, and I'd rather not get dragged into a "yes it does, no it doesn't" match, is there literature or something I can point to that has sufficient basis in "real" applications? I can't be the only one dealing with this, so what do you guys do? --Geoff
Re: [OT] TDD only works for simple things...
I don't know of examples off-hand but I think in a way they're correct. If you write lots of code first and then try to test it, you will look and say "it's not possible to test this so I could not possibly have written my tests beforehand - those TDD guys are fools". If you write the tests beforehand (or even if you just write your code with an eye towards how it will be tested) you end up designing your systems so that even the biggest most complex pieces are testable. So until you actually get the testing bug, it's true that only your simplest designs are testable. Also, the problem with php (assuming you use it as a webpage generator) is that it encourages you to embed code in your HTML and so yes, it is naturally difficult to test, F On 3/28/06, Geoffrey Young <[EMAIL PROTECTED]> wrote: > hi all :) > > for those interested in both php and perl, it seems that php's native .phpt > testing feature will soon produce TAP compliant output - see greg beaver's > comments here > > http://shiflett.org/archive/218#comments > > so, TAP is slowly dominating the world... but we all knew that already :) > > what actually prompted me to write is a comment embedded there: > > "Only the simplest of designs benefits from pre-coded tests, unless you have > unlimited developer time." > > needless to say I just don't believe this. but as I try to broach the > test-driven development topic with folks I hear this lots - not just that > they don't have the time to use tdd, but that it doesn't work anyway for > most "real" applications (where their app is sufficiently "real" or "large" > or "complex" or whatever). > > since I'm preaching to the choir here, and I'd rather not get dragged into a > "yes it does, no it doesn't" match, is there literature or something I can > point to that has sufficient basis in "real" applications? I can't be the > only one dealing with this, so what do you guys do? > > --Geoff >
Re: [perl #38800] Strange dump output from PGE (x86_64 related?)
On Mar 26, 2006, at 16:26, Patrick R. Michaud wrote: I agree it looks like a GC bug, and not anything x86_64 specific. It was definitely a GC bug (COW copies of constant strings did move string buffers). Fixed r12062. leo
Re: [OT] TDD only works for simple things...
David Cantrell wrote: > Geoffrey Young wrote: > >>> "Only the simplest of designs benefits from pre-coded tests, unless >>> you have >>> unlimited developer time." >> >> needless to say I just don't believe this. > > > Try writing a test suite ahead of time for a graphing library. It's > possible (indeed, it's trivial - just check the md5 hashes of the images > that are spat out against images that you have prepared ahead of time in > some other way) but it would be damnably time-consuming to create those > tests. Consequently, I've not bothered. I throw data at it, and look > at the results. If the results are good I then put an md5 hash of the > image into a regression test. well, ok, I'll agree with you if you look at it that way. but I think tdd ought to happen at much lower level than that - certainly there's more to test than just spitting out an image? you're probably calling several different subroutines in order to generate that image, each of which can be developed using tdd, and each of which gets more and more simple as you get deeper into the application I'd suspect. in general that's where I think the gap really rests for people who think tdd doesn't work. or maybe it's my misunderstanding. but I think there are places much deeper than the end product that warrant testing, and where tdd can succeed and add value no matter how large or complex the code is. --Geoff
Re: [OT] TDD only works for simple things...
Geoffrey Young wrote: "Only the simplest of designs benefits from pre-coded tests, unless you have unlimited developer time." needless to say I just don't believe this. Try writing a test suite ahead of time for a graphing library. It's possible (indeed, it's trivial - just check the md5 hashes of the images that are spat out against images that you have prepared ahead of time in some other way) but it would be damnably time-consuming to create those tests. Consequently, I've not bothered. I throw data at it, and look at the results. If the results are good I then put an md5 hash of the image into a regression test. -- David Cantrell
Re: [OT] TDD only works for simple things...
On Tue, Mar 28, 2006 at 05:47:29PM +0100, David Cantrell wrote: > Geoffrey Young wrote: > > >> "Only the simplest of designs benefits from pre-coded tests, unless you > >> have > >> unlimited developer time." > > needless to say I just don't believe this. > > Try writing a test suite ahead of time for a graphing library. It's > possible (indeed, it's trivial - just check the md5 hashes of the images > that are spat out against images that you have prepared ahead of time in > some other way) but it would be damnably time-consuming to create those > tests. Consequently, I've not bothered. I throw data at it, and look > at the results. If the results are good I then put an md5 hash of the > image into a regression test. Strawman. A graphing library is an obvious example where functional testing should be used prior to automated regression testing. Ben
Re: [OT] TDD only works for simple things...
Moin, On Tuesday 28 March 2006 16:11, Geoffrey Young wrote: > hi all :) > > for those interested in both php and perl, it seems that php's native > .phpt testing feature will soon produce TAP compliant output - see greg > beaver's comments here > > http://shiflett.org/archive/218#comments > > so, TAP is slowly dominating the world... but we all knew that already > :) > > what actually prompted me to write is a comment embedded there: > > "Only the simplest of designs benefits from pre-coded tests, unless you > have unlimited developer time." I am not sure if that proves it or not (need coffee before that), but you might look at the huge testsuite for the (moderately complex) Graph::Easy with at least 300 "pre-coded" tests in ascii.t alone: http://search.cpan.org/src/TELS/Graph-Easy-0.43/t/ Creating these tests takes quite a time, but then, to find a bug you have to have a clear testcase first anyway. And once you have it (breaks with old code, works with new code), you can just move it into the t/in/ directory, create the expected output in t/out/ and increment the counter in ascii.t and be done with it. The hard part is of course that all these tests should have been there from revision 0.01 on already (except the ones testing newly implemented features, but then the testsuite is still far from complete anyway) The one thing that absolutely does _not_ work is creating the output and inspecting it visually. Nobody will be able to view and inspect a few thousand testcases each revision - I did about fourty in one year for Graph::Easy alone. Hope this helps, Tels -- Signed on Tue Mar 28 19:14:13 2006 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. "Now, _you_ behave!" pgpl2G1IFfF27.pgp Description: PGP signature
Re: [PATCH] Forking tests with Test::More
On Tue, Mar 28, 2006 at 06:49:15AM -0600 Chris Dolan wrote:
> On Mar 28, 2006, at 4:55 AM, Tassilo von Parseval wrote:
>
> >>Do you think this might work better, or could be implemented as, a
> >>seperate Test::Fork type module?
> >
> >It certainly could be done. But it would essentially share 90% of its
> >code with Test::Builder.
>
> Umm, subclassing? :-)
In practice that would be the same thing. I'd have to override six
methods, including all the big ones ok(), skip(), todo_skip() and
_ending(). So I'd wind up copying their code verbatimly. Test::Builder
was evidently not written with subclassing in mind.
My patch actually addresses this en-passant as it factors out some code
into methods of its own. Those would be the very methods that would need
overriding.
> >It's simple really: Either my proposed method
> >is robust in which case it can go into Test-Simple. Or it isn't. Then
> >there's no need to implement it as a separate module. :-)
>
> I don't think it is that simple. *IF* there are any bugs in this
> niche feature, then it could break the most popular test module on
> CPAN. Clearly from your proposed patch, any code that needs the fork
> feature of Test::More says so explicitly via the $self->{Forked}
> property. So specifying a subclass instead, like a hypothetical
> Test::More::Forked, should be trivial. I have not yet seen any
> justification for adding this feature to Test::More itself.
Well, the author of Test::More seems to differ on that:
Cheers,
Tassilo
--
use bigint;
$n=71423350343770280161397026330337371139054411854220053437565440;
$m=-8,;;$_=$n&(0xff)<<$m,,$_>>=$m,,print+chr,,while(($m+=8)<=200);
Re: [OT] TDD only works for simple things...
Geoffrey Young wrote: David Cantrell wrote: Try writing a test suite ahead of time for a graphing library. It's possible (indeed, it's trivial - just check the md5 hashes of the images that are spat out against images that you have prepared ahead of time in some other way) but it would be damnably time-consuming to create those tests. Consequently, I've not bothered. I throw data at it, and look at the results. If the results are good I then put an md5 hash of the image into a regression test. well, ok, I'll agree with you if you look at it that way. but I think tdd ought to happen at much lower level than that - certainly there's more to test than just spitting out an image? you're probably calling several different subroutines in order to generate that image, each of which can be developed using tdd, and each of which gets more and more simple as you get deeper into the application I'd suspect. There are lots of bits which *can* be tested and which are (or will be anyway once the design has settled down in my head), but they're all to do with wrangling the data that the user supplies into nice structures. Frankly, they're the easy bits. Those internal methods and those data structures are not for public consumption. In fact, the only methods for public consumption that return anything useful are the constructor and the method which spits out an image. All the other methods will either silently consume data or will die if you pass them a recipe for pie instead of a list of numbers (for example). Unfortunately, the draw() method is the one that's the hardest to write, the one that is the most prone to error, the hardest to debug, and the one where it's hardest to write tests in advance. -- David Cantrell
Re: [OT] TDD only works for simple things...
Moin, On Tuesday 28 March 2006 17:14, Fergal Daly wrote: > I don't know of examples off-hand but I think in a way they're [snipabit] > Also, the problem with php (assuming you use it as a webpage > generator) is that it encourages you to embed code in your HTML and so > yes, it is naturally difficult to test, Well, duh! If you break one of the general rules of coding[0], you have to live with the consequences. Best wishes, Tels 0: DMCADS! - Don't mix code and data, stupid! -- Signed on Tue Mar 28 19:19:53 2006 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. "To be beautiful is enough! If a woman can do that well who should demand more from her? You don't want a rose to sing." -- Thackeray pgpcUHXA0xCQz.pgp Description: PGP signature
Re: [OT] TDD only works for simple things...
Ben Evans wrote: Strawman. Ad hominem. A graphing library is an obvious example where functional testing should be used prior to automated regression testing. Yes. It is one of many such examples. It just happens to be the one I am working on as we speak. -- David Cantrell
Re: [OT] TDD only works for simple things...
Geoffrey Young wrote: > "Only the simplest of designs benefits from pre-coded tests, unless you have unlimited developer time." needless to say I just don't believe this. but as I try to broach the test-driven development topic with folks I hear this lots - not just that they don't have the time to use tdd, but that it doesn't work anyway for most "real" applications (where their app is sufficiently "real" or "large" or "complex" or whatever). Isn't this just a specifications issue? Simple designs are easier to specify fully. More complex designs aren't so easy to specify -- or rather, the time to specify 100% of cases isn't always practical. I would imagine that TDD breaks down if figuring out "correct" behavior takes longer than whipping up something that looks right most of the time and that the end customer will pay for. I would imagine that it also breaks down when testing things at too high a level with too many degrees of freedom. Still, there's no reason that TDD can't be used for the well-defined parts of a big project, or used at the unit level where behavior has fewer degrees of freedom. Then the benefits in terms of positive reinforcement of task completion and the safety net against future breakage still apply. David Golden
Re: Perl 6 on Solaris 10 anyone ?
On Tue, 28 Mar 2006, Stefan Parvu wrote: > Any experiments with Perl6/Parrot and Solaris Express > or Solaris 10 ? Im curious how much the language has > changed and how well the new VM would work in Solaris > ... > > Is there any Solaris x86 or sparc package of the new > Parrot VM ? I'm not aware of any pre-built packages, but the Parrot VM should build and past most of its tests on either Solaris x86 or SPARC. There is no official "Perl6" yet, but there is considerable development in the Pugs project http://www.pugscode.org . In order to run pugs, you'll need the ghc Haskell compiler http://www.haskell.org/ghc/. In order to build ghc, you need a pre-existing ghc, though there are some binary packages available. There is no pre-built Solaris/x86 ghc package. There is one for Solaris/SPARC, but you'll have to install some additional libraries first. (See the ghc download page for details.) Last time I checked, not all those libraries built for me on Solaris 8, so I was never able to get ghc (and hence pugs) up and running. You may have better luck on Solaris 10, or the libraries may have been fixed since then. Hope this helps, -- Andy Dougherty [EMAIL PROTECTED]
Re: [OT] TDD only works for simple things...
On 3/28/06, Tels <[EMAIL PROTECTED]> wrote: > Moin, > > On Tuesday 28 March 2006 17:14, Fergal Daly wrote: > > I don't know of examples off-hand but I think in a way they're > [snipabit] > > Also, the problem with php (assuming you use it as a webpage > > generator) is that it encourages you to embed code in your HTML and so > > yes, it is naturally difficult to test, > > Well, duh! If you break one of the general rules of coding[0], you have to > live with the consequences. Yes but "everything you're doing is wrong, you're stupid" probably isn't the counter-argument that Geoff is looking for :) F > > Best wishes, > > Tels > > 0: DMCADS! - Don't mix code and data, stupid! > > -- > Signed on Tue Mar 28 19:19:53 2006 with key 0x93B84C15. > Visit my photo gallery at http://bloodgate.com/photos/ > PGP key on http://bloodgate.com/tels.asc or per email. > > "To be beautiful is enough! If a woman can do that well who should > demand more from her? You don't want a rose to sing." -- Thackeray > > > >
Re: [OT] TDD only works for simple things...
On 3/28/06, David Cantrell <[EMAIL PROTECTED]> wrote: > Geoffrey Young wrote: > > David Cantrell wrote: > >>Try writing a test suite ahead of time for a graphing library. It's > >>possible (indeed, it's trivial - just check the md5 hashes of the images > >>that are spat out against images that you have prepared ahead of time in > >>some other way) but it would be damnably time-consuming to create those > >>tests. Consequently, I've not bothered. I throw data at it, and look > >>at the results. If the results are good I then put an md5 hash of the > >>image into a regression test. > > well, ok, I'll agree with you if you look at it that way. but I think tdd > > ought to happen at much lower level than that - certainly there's more to > > test than just spitting out an image? you're probably calling several > > different subroutines in order to generate that image, each of which can be > > developed using tdd, and each of which gets more and more simple as you get > > deeper into the application I'd suspect. > > There are lots of bits which *can* be tested and which are (or will be > anyway once the design has settled down in my head), but they're all to > do with wrangling the data that the user supplies into nice structures. > Frankly, they're the easy bits. Those internal methods and those data > structures are not for public consumption. In fact, the only methods > for public consumption that return anything useful are the constructor > and the method which spits out an image. All the other methods will > either silently consume data or will die if you pass them a recipe for > pie instead of a list of numbers (for example). > > Unfortunately, the draw() method is the one that's the hardest to write, > the one that is the most prone to error, the hardest to debug, and the > one where it's hardest to write tests in advance. There are things you can do though. For example you can make draw take a canvas argument. Normally the canvas object would end up producing a gif but when testing, you pass in a canvas which just records all the requested operations. You can then make sure that draw() tried to draw the right number of points, lines, circles, squares etc, that it didn't try to draw outside the boundaries, that it tried to put labels close to the things that were being labelled etc etc. If you were graphing a mathematical function you can check that for each (x, y) that was rendered that abs(f(x)- y) < epsilon for some acceptable epsilon you can also make sure that there was an x for each point visible on the x-axis. Tests like these (particularly the mathematical function graphing tests) can definitely be written in advance, F > > -- > David Cantrell >
Re: [OT] TDD only works for simple things...
On Tuesday 28 March 2006 06:11, Geoffrey Young wrote: > "Only the simplest of designs benefits from pre-coded tests, unless you > have unlimited developer time." "If you think TDD is expensive, try debugging." Greg's comments give me the impression that he thinks TDD means writing a whole pile of tests for the whole project, then starting to write code. -- c
Re: [perl #38760] [BUG] Allow passing "todo => $reason" to Parrot::Test::example_output_(is|like|isnt)
On Monday 20 March 2006 11:16, Bernhard Schmalhofer wrote:
> I have put the calls to example_output_like() and example_output_is() as
> comments into t/benchmarks/benchmarks.t.
>
> # XXX use example_output_is() and example_output_like()
> # This does not work yet WRT to TODO
> # if ( ref $outputs{$_} eq 'Regexp' ) {
> # example_output_like( "examples/benchmarks/$_",
> $outputs{$_}, @todo );
> # }
> # else {
> # example_output_is( "examples/benchmarks/$_", $outputs{$_},
> @todo );
> # }
Here's a patch that seems to fix things for me. Is it better for you,
Bernhard?
-- c
=== lib/Parrot/Test.pm
==
--- lib/Parrot/Test.pm (revision 14488)
+++ lib/Parrot/Test.pm (local)
@@ -32,13 +32,16 @@
The parameter C<$unexpected> is the unexpected result.
The parameter C<$description> should describe the test.
-Any optional parameters can follow. For example, to mark a test as a TODO test (where you know the implementation does not yet work), pass:
+Any optional parameters can follow. For example, to mark a test as a TODO test
+(where you know the implementation does not yet work), pass:
todo => 'reason to consider this TODO'
at the end of the argument list. Valid reasons include C,
C, and so on.
+B you I use a C<$description> with TODO tests.
+
=over 4
=item C
@@ -146,25 +149,28 @@
Compiles and runs the C code, passing the test if a string comparison of
output with the unexpected result is false.
-=item C
+=item C
-Determine the language from the extension of C<$example_f> and
-runs language_output_is().
+Determine the language from the extension of C<$example_f> and runs
+language_output_is(). This I set a description for you, so don't pass
+one.
-=item C
+=item C
-Determine the language from the extension of C<$example_f> and
-runs language_output_like().
+Determine the language from the extension of C<$example_f> and runs
+language_output_like(). This I set a description for you, so don't pass
+one.
-=item C
+=item C
-Determine the language from the extension of C<$example_f> and
-runs language_output_isnt().
+Determine the language from the extension of C<$example_f> and runs
+language_output_isnt(). This I set a description for you, so don't pass
+one.
=item C
-Use within a C block to indicate why and how many test
-are being skipped. Just like in Test::More.
+Use within a C block to indicate why and how many tests to skip,
+just like in Test::More.
=item C
@@ -193,6 +199,7 @@
use strict;
use warnings;
+use vars qw(@EXPORT @ISA);
use Cwd;
use Data::Dumper;
@@ -365,7 +372,8 @@
no strict 'refs';
*{$package.'::'.$func} = sub {
-my ( $code, $expected, $desc, %extra ) = @_;
+local $SIG{__WARN__} = \&report_odd_hash;
+my( $code, $expected, $desc, %extra) = @_;
# Strange Win line endings
convert_line_endings( $expected );
@@ -585,25 +593,29 @@
my $meth = $language_test_map{$func};
if ( my $prefix = $builtin_language_prefix{$language} ) {
+my $level = $builder->level();
+$builder->level( $level + 2 );
my $test_func = "${package}::${prefix}_${meth}";
$test_func->( @remaining );
+$builder->level( $level );
}
else {
# TODO: $language should be the name of the test Module
# that would open the door for Scheme::Test
$language = ucfirst($language);
-# make sure TODO will work, by telling Test::Builder which package
-# the .t file is in (one more than usual, due to the extra layer
-# of package indirection
+# make sure TODO will work, by telling Test::Builder which
+# package the .t file is in (one more than usual, due to the
+# extra layer of package indirection
my $level = $builder->level();
$builder->level(2);
# Load module that knows how to test the language implementation
require "Parrot/Test/$language.pm";
+my $class = "Parrot::Test::${language}";
# set the builder object, and parrot config.
-my $obj = eval "Parrot::Test::${language}->new()";
+my $obj = $class->new();
$obj->{builder} = $builder;
$obj->{relpath} = $path_to_parrot;
$obj->{parrot} = $parrot;
@@ -767,6 +779,29 @@
=cut
+sub report_odd_hash {
+my $warning = shift;
+if ($warning =~ /Odd number of elements in hash assignment/) {
+require Carp;
+my @args = DB::uplevel_args();
+shift @args;
+my $func = ( caller() )[2];
+
+Carp::carp(
+"Odd $func invocation; probably missing description for TODO test"
+
Module::Build and installing in non-standard locations
What's the standard (if any) for how to configure a build script to install specific files (e.g. httpd.conf) in someplace other than the standard Perl library/script/man locations? For example, if my distro contains a bunch of .pm files and .pl files, which go in the "normal" place, and my distro also contains: app_startup.pl httpd.conf and I want those installed as: app_startup.pl => "$some_dir/app-startup.pl", # mode 555 httpd.conf => "$some_other_dir/conf", # mode 444 is there a good standardized way to do this? -M
Re: Module::Build and installing in non-standard locations
Matisse Enzer wrote:
What's the standard (if any) for how to configure a build script to
install specific files (e.g. httpd.conf) in someplace other than the
standard Perl library/script/man locations?
For example, if my distro contains a bunch of .pm files and .pl files,
which go in the "normal" place, and my distro also contains:
app_startup.pl
httpd.conf
and I want those installed as:
app_startup.pl => "$some_dir/app-startup.pl", # mode 555
httpd.conf => "$some_other_dir/conf", # mode 444
is there a good standardized way to do this?
There are a number of ways to do this. The most simple is:
use strict;
use warnings;
use File::HomeDir;
my $conf_dir = File::Spec->catdir( File::HomeDir->my_home, '.Foo' );
use Module::Build;
my $builder = Module::Build->new(
module_name => 'Foo',
license => 'perl',
conf_files => {
'etc/httpd.conf' => 'conf/httpd.conf',
},
install_path => {
'conf' => $conf_dir,
},
);
$builder->add_build_element( 'conf' );
$builder->create_build_script();
__END__
The add_build_element() method is the key to the process. It lets M::B
know it should look for a 'conf_files' property that describes files to
be copied from somewhere in your distribution to a directory in 'blib'.
In the above example, 'etc/httpd.conf' in your distribution root to
'blib/conf/httpd.conf'.
Note that it's a current limitation that the first element of the
directory name in blib must be the same as the name of the build
element, in this case 'conf'.
Next the 'install_path' is looked at for a path to install build
elements of that type. The build elements will be copied from the
directory in 'blib' to the directory given. It is not necessary to
specify the 'install_path' here; it can be specified on the command line
by the user when invoking the install action:
./Build install --install-path conf=/etc/
If specified on the command line it will override the constructor arg.
If it is not specified in either place, those files will not be
installed, so it can be used as a flag.
Since we use ExtUtils::Install to install there is currently no way to
specifiy the file mode.
This has turned out to be one of the most FAQ with Module::Build. There
are plans to greatly simplify the process in the next version of M::B,
including removing the above mentioned limitations.
There are also several other more advanced ways to accomplish the above
by creating a subclass and providing methods to enumerate the files you
want to install or to create the files to be installed. There are
examples of these in Module::Build::Cookbook.
Also, there is a list dedicated to Module::Build which I've CC'd.
Regards,
Randy.
Re: Module::Build and installing in non-standard locations
On Tue, 28 Mar 2006, Randy W. Sims wrote:
> The add_build_element() method is the key to the process.
Ah-Ha and thank you Very Nice!
I did read the perldoc - but just didn't get that
add_build_element('foo_files')
would make M::B look for a foo_fiels element - and I thought i would have
to provide a process_foo_files() method, I did not realize that I
didn't have to do that part (the documentation seems to say I had to.)
> Since we use ExtUtils::Install to install there is currently no way to
> specifiy the file mode.
I'll find some way to live with that for now :-)
It's mainly an issue for executables.
> This has turned out to be one of the most FAQ with Module::Build. There
> are plans to greatly simplify the process in the next version of M::B,
> including removing the above mentioned limitations.
Cool, and thanks again.
-Matisse
