That's a great idea. I know of no commercial vendors who provide such detailed info.
Bob -----Original Message----- From: Gabor Grothendieck [mailto:ggrothendi...@gmail.com] Sent: Monday, January 26, 2009 7:52 PM To: Muenchen, Robert A (Bob) Cc: R-help@r-project.org Subject: Re: [R] The Quality & Accuracy of R It would be possible to develop tools to develop code coverage statistics quantifying the percent of the code that the tests exercise. On Fri, Jan 23, 2009 at 10:04 AM, Muenchen, Robert A (Bob) <muenc...@utk.edu> wrote: > Hi All, > > > > We have all had to face skeptical colleagues asking if software made by > volunteers could match the quality and accuracy of commercially written > software. Thanks to the prompting of a recent R-help thread, I read, "R: > Regulatory Compliance and Validation Issues, A Guidance Document for the > Use of R in Regulated Clinical Trial Environments > (http://www.r-project.org/doc/R-FDA.pdf). This is an important document, > of interest to the general R community. The question of R's accuracy is > such a frequent one, it would be beneficial to increase the visibility > of the non-clinical information it contains. A document aimed at a > general audience, entitled something like, "R: Controlling Quality and > Assuring Accuracy" could be compiled from the these sections: > > > > 1. What is R? (section 4) > > 2. The R Foundation for Statistical Computing (section 3) > > 3. The Scope of this Guidance Document (section 2) > > 4. Software Development Life Cycle (section 6) > > > > Marc Schwartz, Frank Harrell, Anthony Rossini, Ian Francis and others > did such a great job that very few words would need to change. The only > addition I suggest is to mention how well R did in, Keeling & Parvur's > "A comparative study of the reliability to nine statistical software > packages, May 1, 2007 Computational Statistics & Data Analysis, Vol.51, > pp 3811-3831. > > > > Given the importance of this issue, I would like to see such a document > added to the PDF manuals in R's Help. > > > > The document mentions (Sect. 6.3) that a set of validation tests, data > and known results are available. It would be useful to have an option to > run that test suite in every R installation, providing clear progress, > "Validating accuracy of t-tests...Validating accuracy of linear > regression...." Whether or not people chose to run the tests, they would > at least know that such tests are available. Back in my mainframe > installation days, this step was part of many software installations and > it certainly gave the impression that those were the companies that took > accuracy seriously. Of course the other companies probably just ran > their validation suite before shipping, but seeing it happen had a > tremendous impact. I don't know how much this would add to download, > but if it was too much, perhaps it could be implemented as a separate > download. > > > > I hope these suggestions can help mitigate the concerns so many non-R > users have. > > > > Cheers, > > Bob > > > > ========================================================= > > Bob Muenchen (pronounced Min'-chen), > > Manager, Research Computing Support > > U of TN Office of Information Technology > > Stokely Management Center, Suite 200 > > 916 Volunteer Blvd., Knoxville, TN 37996-0520 > > Voice: (865) 974-5230 > > FAX: (865) 974-4810 > > Email: muenc...@utk.edu > > Web: http://oit.utk.edu/research <http://oit.utk.edu/scc> > > Map to Office: http://www.utk.edu/maps > > Newsletter: http://listserv.utk.edu/archives/rcnews.html > <http://listserv.utk.edu/archives/statnews.html> > > ========================================================= > > > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
