On Tue, Feb 04, 2014 at 07:01:31AM -0800, Colin Yates wrote: > I said that coverage tools answer a specific question; 'how much of my code > is executed when I do this', where 'this' is typically running a set of > tests. People use that answer to infer how 'safe' their system is because > they equate test coverage and safety (which is often a flawed inference).
I certainly don't connotate high coverage with a safe system. No-one said that. I use coverage to answer two questions: * Which lines of code are being hit during testing? * Which lines of code are not being hit during testing? You'll be surprised at how many times I've discovered dead code, needless checks, error handling etc just simply looking at what code was used when. A functional programming language allows you to write code which isn't used. > > In some environments there is so much incidental complexity that these > metrics are hard to calculate by hand (mutating state, deep object > hierarchies etc.). FP has a number of different design decisions which can > significantly reduce that incidental complexity, so if a tool is still > needed maybe the cause is somewhere else - too much coupling/not enough > ignorance etc. > > I think we fundamentally come from different places as I do think you can > trust people and I would choose a couple of decent engineers (although they > are as rare as hen's teeth) without any tools over all the tools in the > world. You present a false dichotomy here. You can have both. Good engineers aren't afraid to use tools where they make sense. > To be clear, I am not saying I don't see the need for code coverage, I am > saying it should be much easier to keep track of code coverage in an FP > system done well primarily due to the wonderfully low level of influence > referential transparency gives you (for example). On the other hand I > absolutely see the need for an automated tool in other environments because > of the implicit complexity. You used the words "should be", therefore I will assume you haven't conducted any scientific studies to show these facts. May we ignore this point? These kinds of things are hard to quantify. > If you thoroughly test all your code when you write it why do you need a > tool to tell you you missed something? This is just so brain-dead stupid. How do you *know* that you thoroughly tested your code? Where do you get these metrics? There are various methods you need to employ before you can even beging to feel comfortable about saying you have 'thoroughly' tested code. Coverage is one method amongst many. > > Again, note I am talking only about calculating test coverage and not about > testing or how much there should be. > > Not sure how many ways I can say the same thing, but let's try one more; I > never said it was Clojure automatically doing anything, I said it is > possible for a good engineer to know the coverage and safety of their > systems themselves in a well designed and implemented system. Some > environments are full of complexity which make it heard, hence the need for > a tool. I am not categorically saying I can't imagine a world where I > would need said tool in a FP system, but my first question would be am I > using a tool to solve a symptom of poor design. You ask me to disprove things like this? Are you kidding? This is just hand-wavey and pulled out of the air. To say a good engineer can calculate complexity themselves is just asking for trouble. The less you rely on human behaviour the closer you get to doing science. > > In terms of analysing a new system? When I was a consultant reviewing > other's work the best tool I used was a whiteboard, a pen and their > architect. And what happens to this data? Did you store it? Did you write it down? Could you pass that data onto other people easily? It seems that institutional knowledge such as this is a bigger cause for concern than simply using coverage tools. > I found that if their system *needed* a coverage tool the tests were > probably so poorly written as to add very little value. At this point I'm beginning to suspect you've never really used a coverage tool properly. Coverage is used in conjunction with other methods to gain knowledge about your system, if you rely on coverage alone (and don't act on the results so long that you *continually* need it) then you have bigger problems than poor tests. > > I would genuinely like you/others to prove/disprove these points as this is > an area I am still thinking/learning about (as evidenced by my first and > last sentence in the original post) and would love to have a useful > discussion. You haven't bought anything to the table other than little > jibes and emotive statements unfortunately. > > Let's agree to disagree, and if you can resist having a dig on a public > forum (feel free to continue over personal email) let's draw this to a > close. > > On Tuesday, 4 February 2014 14:30:29 UTC, Aaron France wrote: > > > > I took issue with you maintaining that Clojure automatically somehow > > gives you insight into the coverage of your tests. Which it does not. > > > > You still maintain this. > > > > On Tue, Feb 04, 2014 at 06:28:51AM -0800, Colin Yates wrote: > > > I have no idea why you aren't gushing. I'm not gushing, and haven't > > gushed > > > about anything technical for years because everything is a trade off and > > > has its own compromises/ceremony. I can see (and highly value) the > > > benefits of Clojure, sure. > > > > > > If you want to write of my point of view as 'gushing' and not bother to > > > read it correctly then fine. However, what is your objective in posting > > > your statement to a public forum if not to start an argument? > > > > > > If you insist on sending more flame bait/trying to get a rise then let's > > > take this offline and keep this list low noise. My email address is > > colin > > > full stop yates @ Google's mailing servers.com. > > > > > > On Tuesday, 4 February 2014 14:17:25 UTC, Aaron France wrote: > > > > > > > > I don't come from 'Java-land'. I'm primarily an Erlang developer, > > > > which already is a very similar language to Clojure. Perhaps this is > > > > why I'm not gushing about functional programming's panacea? > > > > > > > > Aaron > > > > > > > > On Tue, Feb 04, 2014 at 06:12:18AM -0800, Colin Yates wrote: > > > > > This has turned into an unconstructive argument and for whatever > > reason > > > > we > > > > > don't seem to be communicating clearly. Shame as I (and probably > > most > > > > > people on here) only want to help. You seem to be reacting quite > > > > strongly > > > > > to my thoughts - not sure why. > > > > > > > > > > If I may, I will just make/rephrase two points: > > > > > - I think you would find value in watching Rick Hickey's videos on > > > > "Simple > > > > > Made Easy" and also the one where he talks about "Hammock Driven > > > > > Development". > > > > > - when I started using Clojure I immediately looked for equivalents > > of > > > > all > > > > > the supporting infrastructure I used in good old Java land. I have > > no > > > > idea > > > > > of your situation, but if you are there you have a wonderful > > opportunity > > > > to > > > > > re-examine and build up a whole new toolchain/approach to > > development > > > > that > > > > > IME is significantly lighter and more powerful. > > > > > > > > > > Peace. > > > > > > > > > > On Tuesday, 4 February 2014 13:49:49 UTC, Aaron France wrote: > > > > > > > > > > > > On Tue, Feb 04, 2014 at 04:18:30AM -0800, Colin Yates wrote: > > > > > > > Comments in line. > > > > > > > On Tuesday, 4 February 2014 11:23:36 UTC, Aaron France wrote: > > > > > > > > > > > > > > > > > > > > > > > > I don't want to seem rude but I think you've drank a bit too > > much > > > > > > > > kool-aid. > > > > > > > > > > > > > > > You know the phrase "I don't want to seem rude" doesn't actually > > do > > > > > > > anything right? :) > > > > > > > > > > > > > > > > > > > > > > > > > > I genuinely don't want to offend. People allow themselves to > > become > > > > > > vested in their viewpoint. If that has happened to you, I'm sorry. > > > > > > > > > > > > > > To say that functional programming and war against state means > > > > that > > > > > > > > your application doesn't need to be tested thoroughly is a > > joke. > > > > And a > > > > > > > > very bad one. > > > > > > > > > > > > > > > I agree, but who is saying that? I certainly didn't cover how > > much > > > > > > testing > > > > > > > is necessary. I thoroughly test my Clojure systems using midje, > > > > which > > > > > > > regularly rocks my world. My point is that the coverage is much > > > > *much* > > > > > > > easier to reason about in FP than in OO (for the reasons I > > gave). > > > > > > > > > > > > I'm not following how you translate this into information which > > > > > > explains how your system is being tested. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Coverage doesn't just aid you in seeing which parts of state > > > > caused > > > > > > > > which branches to be hit, it also gives you notice if there > > are > > > > any > > > > > > > > logical errors in your code which cause the branches to not be > > > > hit. > > > > > > > > > > > > > > > And why are those logical errors which cause the branches to not > > be > > > > hit > > > > > > not > > > > > > > immediately obvious? Why do you need a tool to tell you that? > > I > > > > know > > > > > > my > > > > > > > Clojure code has around 100% coverage using white box testing > > for > > > > the > > > > > > > functions and mocking the interactions. > > > > > > > > > > > > And what's the harm in getting this information from an automated > > > > > > tool? With your 20 years industry knowledge you should know that > > you > > > > > > cannot rely on humans to think and act reliably. It's just not a > > good > > > > > > way to plan systems. *Especially* when it comes to asking someone > > how > > > > > > correct their system is. > > > > > > > > > > > > > > I would challenge you to put ego/emotion to one side, stop > > finding > > > > > > > non-existent points to argue against and re-read my post. By > > all > > > > means > > > > > > > come back and justify why all the points I raised which reduce > > the > > > > need > > > > > > for > > > > > > > coverage are invalid. Don't attribute stupid statements (like > > 'FP > > > > > > doesn't > > > > > > > need testing') to me - I can come up with my own stupid > > statements > > > > thank > > > > > > > you. > > > > > > > > > > > > You hand waved the need to use tools such as coverage reports > > simply > > > > > > on the virtue of some hard to quantify statements. I find that > > > > > > unscientific. > > > > > > > > > > > > > > If it helps, my stand point is from 20 years of building > > non-trivial > > > > > > > Enterprise applications (primarily Java) using the current best > > of > > > > breed > > > > > > > technology stacks (i.e Spring/Hibernate/AspectJ) with the best > > of > > > > breed > > > > > > > process (agile, TDD, DBC, BDD, most otherTLAs etc.). > > > > > > > > > > > > Arguments from authority mean nothing on the internet. > > > > > > > > > > > > > Using Clojure for the past year or so has opened my eyes to > > exactly > > > > > > > how many problems we solve, and infrastructure we use is to > > pamper > > > > > > > to complexity introduced by the tool-chain not the problem > > domain. > > > > > > > I am suggesting maybe coverage tools are one of those. > > > > > > > > > > > > > > > > > > > Coverage helps nothing on its own. It's a tool to aid in knowing > > which > > > > > > aspects of your system remain untested. It's fine to *believe* > > you're > > > > > > testing 100% of your system, but how do you actually know this? > > > > > > > > > > > > If you wander into a codebase you're not familiar with, what's the > > > > > > coverage? How do you know you're hitting all codepaths? You just > > > > > > cannot know this without reading all the code and the tests. > > Coverage > > > > > > helps to discover this information. > > > > > > > > > > > > My point isn't to eschew all other forms of testing in favour of > > > > > > coverage reports but to use them in tandem with the others to aid > > me > > > > > > in *knowing* which parts of the system are being tested and which > > are > > > > not. > > > > > > > > > > > > Aaron > > > > > > > > > > > > > > > > > > > > > Aaron > > > > > > > > > > > > > > > > On Tue, Feb 04, 2014 at 03:19:05AM -0800, Colin Yates wrote: > > > > > > > > > I don't know. > > > > > > > > > > > > > > > > > > But maybe the lack of coverage tools is itself interesting? > > My > > > > (not > > > > > > > > quite > > > > > > > > > formed/making this up as I go) view is that maybe coverage > > tools > > > > are > > > > > > > > there > > > > > > > > > to address the implicit complexity in other mainstream > > languages > > > > > > and/or > > > > > > > > to > > > > > > > > > help mitigate the risk of the potentially large and > > > > hard-to-identify > > > > > > > > > 'impact analysis' you get in OO systems when you change > > state. > > > > In > > > > > > other > > > > > > > > > words, coverage is necessary because we want to feel safe > > that > > > > all > > > > > > > > > combinations of our code are extensively tested. Why don't > > we > > > > feel > > > > > > > > safe? > > > > > > > > > Because the system is hard to reason about. > > > > > > > > > > > > > > > > > > Functional programming on the other hand is full of much > > smaller > > > > > > > > discrete > > > > > > > > > and independent chunks of functionality. Ideally these > > small > > > > > > focused > > > > > > > > > 'bricks' are pure/referentially transparent so the *only* > > > > context > > > > > > you > > > > > > > > need > > > > > > > > > when reasoning about them is their parameters and the logic > > > > inside. > > > > > > > > > Assembling these bricks introduces interactions that need > > to be > > > > > > tested, > > > > > > > > > sure, but there are very few 'call this and watch the change > > > > > > > > cascade'/'this > > > > > > > > > code is sensitive (i.e. coupled) to that data over there'. > > > > > > > > > > > > > > > > > > My ramblings are to say, maybe the root cause of coverage > > tools > > > > is > > > > > > to > > > > > > > > solve > > > > > > > > > a problem (hard to reason about systems) which shouldn't be > > much > > > > > > less of > > > > > > > > a > > > > > > > > > problem in FP when FP is done right. OO + mutable state = > > hard > > > > to > > > > > > > > reason > > > > > > > > > about. FP + immutable state + pure/referentially > > transparent > > > > > > functions > > > > > > > > = > > > > > > > > > much easier to reason about. > > > > > > > > > > > > > > > > > > Or not. Just my 2 pence :). > > > > > > > > > > > > > > > > > > On Sunday, 2 February 2014 21:34:29 UTC, Aaron France wrote: > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > I'm looking for coverage reporting in Clojure. I've been > > using > > > > > > > > > > Cloverage[1] but I'm just wondering if there are any other > > > > > > coverage > > > > > > > > > > tools? > > > > > > > > > > > > > > > > > > > > Aaron > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [1] https://github.com/lshift/cloverage > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > You received this message because you are subscribed to the > > > > Google > > > > > > > > > Groups "Clojure" group. > > > > > > > > > To post to this group, send email to > > > > > > > > > clo...@googlegroups.com<javascript:> > > > > > > > > > > > > > > > > > > > > > Note that posts from new members are moderated - please be > > > > patient > > > > > > with > > > > > > > > your first post. > > > > > > > > > To unsubscribe from this group, send email to > > > > > > > > > clojure+u...@googlegroups.com <javascript:> > > > > > > > > > For more options, visit this group at > > > > > > > > > http://groups.google.com/group/clojure?hl=en > > > > > > > > > --- > > > > > > > > > You received this message because you are subscribed to the > > > > Google > > > > > > > > Groups "Clojure" group. > > > > > > > > > To unsubscribe from this group and stop receiving emails > > from > > > > it, > > > > > > send > > > > > > > > an email to clojure+u...@googlegroups.com <javascript:>. > > > > > > > > > For more options, visit > > https://groups.google.com/groups/opt_out. > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the > > Google > > > > > > > Groups "Clojure" group. > > > > > > > To post to this group, send email to > > > > > > > clo...@googlegroups.com<javascript:> > > > > > > > > > > > > > Note that posts from new members are moderated - please be > > patient > > > > with > > > > > > your first post. > > > > > > > To unsubscribe from this group, send email to > > > > > > > clojure+u...@googlegroups.com <javascript:> > > > > > > > For more options, visit this group at > > > > > > > http://groups.google.com/group/clojure?hl=en > > > > > > > --- > > > > > > > You received this message because you are subscribed to the > > Google > > > > > > Groups "Clojure" group. > > > > > > > To unsubscribe from this group and stop receiving emails from > > it, > > > > send > > > > > > an email to clojure+u...@googlegroups.com <javascript:>. > > > > > > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > > > > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "Clojure" group. > > > > > To post to this group, send email to > > > > > clo...@googlegroups.com<javascript:> > > > > > > > Note that posts from new members are moderated - please be patient > > with > > > > your first post. > > > > > To unsubscribe from this group, send email to > > > > > clojure+u...@googlegroups.com <javascript:> > > > > > For more options, visit this group at > > > > > http://groups.google.com/group/clojure?hl=en > > > > > --- > > > > > You received this message because you are subscribed to the Google > > > > Groups "Clojure" group. > > > > > To unsubscribe from this group and stop receiving emails from it, > > send > > > > an email to clojure+u...@googlegroups.com <javascript:>. > > > > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "Clojure" group. > > > To post to this group, send email to clo...@googlegroups.com<javascript:> > > > Note that posts from new members are moderated - please be patient with > > your first post. > > > To unsubscribe from this group, send email to > > > clojure+u...@googlegroups.com <javascript:> > > > For more options, visit this group at > > > http://groups.google.com/group/clojure?hl=en > > > --- > > > You received this message because you are subscribed to the Google > > Groups "Clojure" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > an email to clojure+u...@googlegroups.com <javascript:>. > > > For more options, visit https://groups.google.com/groups/opt_out. > > > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out.
pgpqmxG1V2YYp.pgp
Description: PGP signature
