Hi.
On Sat, 14 May 2016 15:05:54 -0500, Matt Sicker wrote:
3. Hardware bugs? Rare but possible.
The behaviour is the same on two different machines.
As it is blocking MATH-1341, I can do one the following:
1. annotate the test with "@Ignore"
2. find a seed that will make the test pass
3. remove the assertion that fails
At first sight, I'd think that the former is a little safer
as it leaves a trace that something might not work as intended
(or if it is expected, that the reason for it to fail so often
must be documented).
Please advise.
Regards,
Gilles
On 14 May 2016 at 08:01, Gilles <[email protected]> wrote:
On Sat, 14 May 2016 02:47:18 +0200, Gilles wrote:
Hello.
Test method "testNaNsFixedTiesRandom" (in the above unit test
class)
can pass or fail depending on the seed value.
When seed is 1000, the test passes.
When seed is 1001, this failure occurs:
Elements at index 1 differ. expected = 3.0 observed = 4.0
Elements at index 4 differ. expected = 3.0 observed = 2.0
When seed is 1002, this failure occurs:
Elements at index 1 differ. expected = 3.0 observed = 2.0
Elements at index 4 differ. expected = 3.0 observed = 4.0
Elements at index 8 differ. expected = 2.0 observed = 3.0
When seed is 1003, this failure occurs:
Elements at index 8 differ. expected = 2.0 observed = 4.0
When seed is 1004, this failure occurs:
Elements at index 1 differ. expected = 3.0 observed = 2.0
When seed is 1005, this failure occurs:
Elements at index 4 differ. expected = 3.0 observed = 2.0
Elements at index 8 differ. expected = 2.0 observed = 3.0
When seed is 1006, this failure occurs:
Elements at index 1 differ. expected = 3.0 observed = 4.0
Elements at index 4 differ. expected = 3.0 observed = 4.0
Elements at index 8 differ. expected = 2.0 observed = 3.0
When seed is 1007, this failure occurs:
Elements at index 1 differ. expected = 3.0 observed = 2.0
Elements at index 4 differ. expected = 3.0 observed = 4.0
When seed is 1008, this failure occurs:
Elements at index 1 differ. expected = 3.0 observed = 2.0
Elements at index 8 differ. expected = 2.0 observed = 4.0
When seed is 1009, this failure occurs:
Elements at index 1 differ. expected = 3.0 observed = 2.0
When seed is 1010, this failure occurs:
Elements at index 1 differ. expected = 3.0 observed = 4.0
Elements at index 4 differ. expected = 3.0 observed = 2.0
Elements at index 8 differ. expected = 2.0 observed = 3.0
Also fails when seed is
1011
1012
1013
1014
1015
1016
1017
112351341
-932524
Is that expected behaviour?
It does not look trivial to understand why one should trust a
test that fails most of the time...
The test fails for ~96.3% of the possible seed values.
Cause for such a behaviour can be:
1. unit test is buggy
2. code being tested is buggy
Are there other possible causes?
Regards,
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
