Kovas is right on the money here. Inherently when you make something ‘programmable’ generating correct programs for it is hard.
I would say though, I frequently see people struggling to spec more complex logic are going directly to independent input generation. Thus, how will inputs be reasonably coordinated etc. The better approach is to first generate a consistent model, then generate inputs from that model. gen/bind is your friend. Typically the approach is: (gen/bind model-generator (fn [model] data-generator)) Each model then need not be exhaustively large because many models will be generated, and the model lets you share things between what would otherwise be independent generators. On the specific points: > 1. No way to test function output specs. For documentation purposes, I want > to have an output spec on my function. However, as far as I know, after > instrumentation-triggered checking of output specs was removed a couple of > alphas ago, the only way remaining to check against output specs is to use > randomly generated tests. So if I can't make good generators, I have no way > to confirm that my output spec works the way I think it does. My > documentation could be totally out of touch with reality, and that displeases > me. Running return-value instrument-style checking on whatever few hand-written tests you might have isn’t going to give you better coverage than a simple (even hardwired) generator that captures similar ranges. And you can directly exercise your :ret spec - it’s just another spec. You can also spec/assert on your return - the tests will disappear in production, although this is similarly as weak as instrument-triggered return checking, so I’m not going to help you do it, but you can do it yourself :) > 2. Limited ability for testing that functions take and receive what you > expect. Whereas a static type system can prove certain properties about > whether functions are called with valid inputs, with spec, you only get those > guarantees if you pump a function with enough valid random data to trigger > the function calling all its callees with interesting combinations of data. > But if I use the naive generators, the function will never even complete with > most of the randomly generated input, let alone call other functions in a > useful way. And in many cases I don't see how to generate something of > better quality. You can certainly use core.typed if you want to check the kinds of things it can check in the way it can check them. But saying specs aren’t types says little about either, and using the word ‘valid’ in both contexts without qualification equates things that are not the same. It’s not as if most type systems can check the predicates spec can (Idris et al aside). There’s a big difference between type-correct inputs/outputs and stakeholder-correct programs. spec is oriented towards the latter, but, the bar, as you have seen, is generation. It is, however, a local challenge - ‘this particular fn is hard to gen for’. The bar for static typing is flowing types everywhere, a bar I consider to be more global, harder to meet, and less expressive. > One other issue I've had, unrelated to generators, is that I'm struggling to > express higher-order type constraints, for example, this function takes a > vector v1 of anything and a vector v2 of anything, but the type of the things > in vector v1 better match the type of the things in vector v2. Parameterized types definitely have advantages for the subset of things they can check in this regard. But again, it is a logical fallacy to equate spec == type then switch to static type checking ‘advantages' as if they had the same expressive power - in general they don’t, since specs are predicative and of runtime values. Most type systems can’t tell you that the things in v1 satisfy the same predicates as do the things in v2 for any predicate other than ‘is statically a T’ (Idris et al aside). That said, I have been thinking about the ‘parameterized spec’ problem, nothing concrete to show yet. Rich -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
