I'm pretty sure I did encounter the performance problem you're talking about,
but I killed it and re-ran until I hit cases that shrink quickly. I'm afraid
I'm not much help with those, although I agree that the bad shrinking is
probably related to the performance issues.
> On 13 Nov 2014, at 04:22, Brian Craft <craft.br...@gmail.com> wrote:
>
> Interesting that you don't see a performance problem. What version did you
> try? I'm using 0.5.9.
>
> I just re-ran this example to make sure I wasn't imagining it. On the 11th
> run, it wedged, not returning, and burning a lot of cpu, presumably trying to
> shrink.
>
> It's a larger problem with the real test cases. One test ran for about eight
> hours yesterday before I killed it. Hard to tell what it's doing, and not
> sure what sort of times I should expect for shrinking larger test cases.
>
>
>> On Wednesday, November 12, 2014 9:05:40 AM UTC-8, Lucas Bradstreet wrote:
>> I've also had some tricky shrinking type issues with recursive generators
>> using bind. I had a play with your generators, using such-that to reduce the
>> row/column name length and also preventing some generator shrinking by using
>> no-shrink, but I didn't have much luck improving the resulting shrinks
>> (though the shrinks did finish in a timely fashion).
>>
>> You might have more luck if you generated an m x n matrix size independently
>> of the matrix itself and feed the size into the matrix generator and the
>> samples and probes generators. This may allow the matrix size to be shrunk
>> down more easily, while also shrinking the samples and probes vectors
>> without them having to be regenerated for ever new matrix shrink (i.e. where
>> the matrix size is maintained but the gen/int values within it are shrunk).
>>
>> However, this is all guess work and conjecture, as I don't understand the
>> shrinking algorithm well enough, and I haven't tried my suggestion. Also,
>> modifying your generators in this way may make your model less general if
>> you wish to use it in other ways later.
>>
>> Cheers
>>
>> Lucas
>>
>>
>>
>>> On 12 Nov 2014, at 02:43, Brian Craft <craft...@gmail.com> wrote:
>>>
>>> Using test.check, I'm finding the shrinking to be very, very slow. Running
>>> a hundred cases takes a few seconds, unless it hits an error, in which case
>>> it takes 40-60 minutes to shrink, and the shrinking is not very effective
>>> (resulting test case is much larger than necessary). Sometimes the
>>> shrinking is much faster. It behaves a bit like it's occasionally getting
>>> into a pathological state, or a difficult shrinking scenario.
>>>
>>> Are there any docs on generators or the shrinking algorithm that would help
>>> build tests that shrink more effectively?
>>>
>>> The problematic generator builds a randomly-sized matrix of integers, with
>>> randomly assigned names for the rows and columns. The failure case is when
>>> either a column or row name is repeated. I expect the slow shrinking has
>>> something to do with it being rare for the generator to emit the same name
>>> twice.
>>>
>>> ; Generator of randomly sized matrices of random numbers.
>>> (def gen-matrix
>>> (gen/bind
>>> gen/s-pos-int
>>> (fn [x] (gen/bind
>>> gen/s-pos-int
>>> (fn [y] (gen/vector (gen/vector gen/int x) y))))))
>>>
>>> ; Generator of matrix with probe and sample ids.
>>> (def gen-tsv
>>> (gen/bind
>>> gen-matrix
>>> (fn [m]
>>> (gen/hash-map
>>> :probes (gen/vector
>>> (gen/such-that not-empty gen/string-alpha-numeric)
>>> (count m))
>>> :samples (gen/vector
>>> (gen/such-that not-empty gen/string-alpha-numeric)
>>> (count (first m)))
>>> :matrix (gen/return m)))))
>>>
>>> Shrinking will result in a case like
>>>
>>> {:matrix [[1 4 -3] [-4 -3 -5] [-5 2 3] [4 -5 -5] [1 -2 3] [1 4 1]],
>>> :samples ["0" "0" "0"], :probes ["0" "0" "0" "0" "0" "0"]}
>>>
>>> where :samples ["0" "0"] :probes["0"] would do.
>>>
>>> The following test will exhibit the behavior, sometimes succeeding,
>>> sometimes failing quickly, sometimes shrinking for a very long time:
>>>
>>> (tc/quick-check 100 (prop/for-all [tsv gen-tsv] (= (count (set (:probes
>>> tsv))) (count (:probes tsv)))))
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clo...@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+u...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google Groups
>>> "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to clojure+u...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
