On Friday 20 June 2003 20:21, Ken Williams wrote:
> Second, I find it very confusing that all these different capabilities
> are happening inside one cmp_deeply() function. In Perl it's much more
> common to use the function/operator to indicate how comparisons will be
> done - for example, <=> vs. cmp, or == vs. eq. I would much rather see
> these things broken up into their own functions.
I had a hard time trying to document this module and I wasn't sure I did a
good job, now I'm certain I didn't! I hope I can explain in this email. It's
a bit long but I hope you will see at the end that your comments are based on
a misunderstanding of what Test::Deep does. I'd really appreciate it if you
could tell me if it makes any sense to you or if it makes no sense at all I
don't want to alienate users just because my docs are unintelligible. As a
bonus, since you're @mathforum.org I'll throw in some non-well-founded set
theory near the end ;-)
First off, the Test::Deep functions set(), bool(), bag(), re() etc are not
comparison functions, they are shortcuts to Test::Deep::Set->new,
Test::Deep::Bool->new, Test::Deep::Bag->new, Test::Deep::Regex->new. The
objects they return act as markers to tell Test::Deep that at this point of
the comparison to stop doing a simple comparison and to hand over control to
Test::Deep::Whatever.
There's nothing you can do in regular expression that you can't do with substr
and eq but regular expressions allow you to express complex tests in a simple
form. That is the goal of Test::Deep. Perl has regexs that operate on a
linear array of characters, Test::Deep supplies "regular expressions" that
operate on an arbitrarily complicated graph of data and just as a regex often
looks like the strings it will match, a Test::Deep structure should look like
the structure it will match.
What's wrong with using Test::More::is_deeply()? Well, is_deeply is just the
complex-structure equivalent of eq for strings. is_deeply checks that two
data structures are identical. What do you do if part of part of the
structure you're testing is unpredictable? Maybe it comes from an outside
source that your test script can't control, maybe it's an array in an
undetermined order or maybe it contains an object from another module - you
don't want your test to look inside other modules' objects because you have
no way of telling if it's right or wrong. In these cases is_deeply() will
fail and so is no use. Test::Deep::cmp_deeply() has varying definition of
"equality" and so can perform tests that is_deeply can't.
Time for some examples.
Simple string case: Say you want to test a string that is returned from the
function fn(). You know it should be "big john". So you do
Test::More::is(fn(), "big john", "string ok");
Messy string case: Things change, now fn() returns a string that contains "big
john" and some other stuff, you can't be sure what the other stuff is, all
you know is that the string should be a number, followed by "big john",
possibly followed by some other stuff. No problem
Test::More::like(fn(), qr/^\d+big john.*/, "string ok");
Now imagine that you have a function that returns a hash
Simple structure case: you want to test that fn() returns
{
age => 34,
id => "big john",
cars => ['toyota', 'fiat', 'citroen'],
details => [...] # some horrible complicated object
}
Test::More::is_deeply(fn(),
{
age => 34,
id => "big john",
cars => ['toyota', 'fiat', 'citroen'],
details => [...] # some horrible complicated object
}
);
Messy structure case: same as above but say now the id is no longer simply
"big john", it's the same messy thing we talked about in the messy string
case, and say you're no longer guaranteed that the cars will come back in any
particular order because they're coming from an unorderd SQL query.
Test::is_deeply is no good now as it needs exact equality. You could write
my $hash = fn();
is($hash->{age}, 34);
like($hash->{id}, qr/^\d+big john.*/);
is_deeply(sort @{$hash->{cars}}, ['citroen', 'fiat', 'toyota' ]);
is_deeply($hash->{details}, [...]);
is(scalar keys %$hash, 4);
but you'd be soooooooo wrong because you've also got to check that all your
refs are defined before you go derefing them so here's the full ugliness you
really need
if( is(Sclar::Util::reftype($hash), "HASH") )
{
is($hash->{age}, 34);
like($hash->{id}, qr/^\d+big john.*/);
if( is(Sclar::Util::reftype($hash->{cars}), "ARRAY") )
{
is_deeply(sort @{$hash->{cars}}, ['citroen', 'fiat', 'toyota' ]);
}
else
{
fail("no array");
}
if( is(Sclar::Util::reftype($hash->{details}), "ARRAY") )
{
is_deeply($hash->{details}, [...]);
}
else
{
fail("no array");
}
}
else
{
for (1..6) # cos we don't want to mess up the plan!
{
fail("no hash");
}
}
Notice the complete lack of test names as by now you've lost the will to test
properly.
Test::Deep to the rescue!!
Test::Deep::cmp_deeply($hash,
{
age => 34,
id => re(qr/^\d+big john.*/),
cars => bag('toyota', 'fiat', 'citroen'),
details => [...] # some horrible complicated object
},
"check big john"
);
You'll get informative error messages if $hash is wrong and your test will
never explode because of undefined values.
Here's a walk through how the comparison works, rather than refererring to the
structure above as "the second argument to cmp_deeply", I'll call if $exp
(short for expected). So
$exp = {
age => 34,
id => re(qr/^\d+big john.*/), # Test::Deep::Regex constructor
cars => bag('toyota', 'fiat', 'citroen'), # Test::Deep::Set constructor
details => [...] # some horrible complicated object
};
cmp_deeply($hash, $exp, "check big john");
works like this
- call descend($hash, $exp)
- $exp is a HASH ref and so is $hash so call descend_hash($exp,$hash)
- get a key from $exp - "age"
- call descend($hash->{"age"}, $exp->{"age"})
- $hash->{"age"} and $exp->{"age"} are both scalars
- check $hash->{"age"} eq $exp->{"age"}
- get next key from $exp - "id"
- call descend($hash->{"id"}, $exp->{"id"})
- $exp->{id} is a Test::Deep::Regex object, this means we don't just do
a simple eq, instead we hand control over to the Test::Deep::Regex module by
calling $exp->{id}->descend and passing in $hash->{id}
- what happens inside Test::Deep::Regex::descend is a mystery that
Test::Deep knows nothing about, all it wants is a return code of 1 or 0. It
get a 1, and breathes a sigh of relief
- get next key from $exp - "cars"
- call descend($hash->{"cars"}, $exp->{"cars"})
- $exp->{id} is a Test::Deep::Set object so again call it's descend
method
- the Set comparison was OK
- get next key from $exp - "details"
- call descend($hash->{"details"}, $exp->{"details"})
- $hash->{"details"} and $exp->{"details"} are both ARRAY refs so call
descend_array
- much descending and comparing
- there is no next key in $exp, did we use up all the keys in $hash? Yes
All stages of the test passed, so it's a pass. If it had failed, then you get
dagnostics like
# checking $data->{"age"}
# expected: "34"
# got: "27"
or
# checking $data->{"id"} as a regex
# expected: something matching qr/\d+big john.*/
# got: "14556big mick"
Other types of comparison are
bool - make sure something is true or false, without caring exactly what it is
ignore - don't look any further into this part of the structure
isa - check that it's a blessed ref from this class
methods - call these various methods and make sure they return these values
shallow - check that it is exactly this reference rather than delving into it
and several more, with more to come. Plus you can define your own and even
export them for other modules to use.
So I hope you can see now why there is only 1 comparison function and why all
the other functions can't be "broken up".
Now for the set theory. Imagine 3 sets A, B and C and 3 items x, y, z. Here's
what our sets look like
A = (B, C, x)
B = (A, C, y)
C = (A, B, z)
In traditional set theory this would be illegal. There is the axiom of
foundation which roughly says that a set S cannot be an element of itself or
of any of it's elements or of any of their elements or ... Basically S can't
appear inside S, now matter how deeply buried. Some really smart guy proved
that this axiom is independent of the other axioms of set theory. This means
you can assume it's true and you get one set of theorems or you can assume
it's false and you get another different set of equally valid theorems. This
is good thing, otherwise
$a = [$a];
would be probably make your computer explode.
What's this to do with Test::Deep? Due to Test::Deep working the way it does
and being able to handle circular refs you get and almost useless but, I
think, quite cool feature. Test::Deep is able to test nasty non-wellfounded
set comparions. So for the example above:
my $A = set();
my $B = set();
my $C = set();
$A->add("x", $B);
$B->add("y", $C);
$C->add("z", $A);
$A->add($C);
$B->add($A);
$C->add($B);
# that took a bit longer than it should have due to an "issue" I just found
my $D = [];
my $E = [];
my $F = [];
push(@$D, "x", $E, $F, "x", $E, $F);
push(@$E, $D, $D, "y", $F);
push(@$F, $E, $D, $D, "z");
cmp_deeply($D, $A); # because dups and order don't matter this is a yes
push($F, $F);
# now we have
# F = (D, E, F, "z") instead of (D, E, "z")
cmp_deeply($D, $A); # no although it takes a little while to decide
Please let me know whether this made any sense. If you have any suggestions
how I can explain Test::Deep in 20 words, I'd really appreciate it!
F
