Duncan, you are right that from the perspective of a single developer: it is entirely possible to make tests conditional and ensure that 'long' tests does not get run by CRAN.
What I, and several others, advocate is a uniform set of conventions, so we avoid having multiple package specific solutions. Martin has explained one reason for why using _R_CHECK_TIMINGS_ is not ideal (the fact that we - at least I and Martin - are most interested in the timings for the long tests). Another reason is that I (and I suspect most others) would prefer something that looks a bit more official and supported. High-jacking an environment variable set for (what appears to be) a different reason does not seem like something that could be considered supported long-term, and which have a uniform interface (across packages). Of course, adding an extra environment variable or an extra function (as suggested by Henrik) is additional work, and I know it is easy for us to suggest things. My perspective is that I would like something that will continue to be supported in the future, and where I can use the same interface across the packages I maintain and others outside my control that I occasionally test. Testing is not just being done by the package author(s) or CRAN. I suspect many packages already have half-baked solutions to the 'long' test vs 'short' test issue, and I don't think we will see any movement toward a unified interface without support in base R. And let me end by saying that I do agree with Martin: having numerical levels is better (together with a clear statement what --as-cran sets the level to). But I would rather have a binary state than nothing. Kasper On Wed, Sep 5, 2012 at 9:10 AM, Duncan Murdoch <murdoch.dun...@gmail.com> wrote: > This post contains two incorrect statements. Since Martin is too busy to > post a retraction, I will point them out: > > --timings and --as-cran do not set _R_CHECK_TIMINGS_ in the same way. > > As far as I recall, nobody has suggested that package writers be limited to > two choices for test suites. I certainly haven't. > > Duncan Murdoch > > > On 05/09/2012 6:25 AM, Martin Maechler wrote: >> >> >>>>> Deepayan Sarkar <deepayan.sar...@gmail.com> >> >>>>> on Wed, 5 Sep 2012 11:49:37 +0530 writes: >> >> > On Wed, Sep 5, 2012 at 11:41 AM, Duncan Murdoch >> > <murdoch.dun...@gmail.com> wrote: >> >> On 12-09-04 8:19 PM, Kasper Daniel Hansen wrote: >> >>> >> >>> On Tue, Sep 4, 2012 at 6:02 PM, Duncan Murdoch >> <murdoch.dun...@gmail.com> >> >>> wrote: >> >>>> >> >>>> On 04/09/2012 5:42 PM, Dirk Eddelbuettel wrote: >> >>>>> >> >>>>> >> >>>>> On 4 September 2012 at 17:26, Duncan Murdoch wrote: >> >>>>> | On 04/09/2012 5:14 PM, Dirk Eddelbuettel wrote: >> >>>>> | > An add-on argument to the already established option >> --as-cran may >> >>>>> be >> >>>>> the >> >>>>> | > best. >> >>>>> | > >> >>>>> | > And to iterate, what bugs me is that for _me_ on _my_ >> machine >> >>>>> developing _my_ >> >>>>> | > package I have remember how to enable what is now (as per >> CRAN's >> >>>>> decree) >> >>>>> | > "non-standard behaviour" of full testing. I fully agree >> with what >> >>>>> Terry had >> >>>>> | > said: more tests are better (when we develop). I want the >> full >> >>>>> suite >> >>>>> at my >> >>>>> | > end; that is after all why we wrote it! >> >>>>> | >> >>>>> | You don't have to remember that, you need to figure it out >> once, write >> >>>>> a >> >>>>> | script that sets the environment variables that enable it, >> and then >> >>>>> you >> >>>>> | can forget it. >> >>>>> >> >>>>> "In theory, theory and practice are the same. In practice, they >> are >> >>>>> not." >> >>>>> >> >>>>> The main test script long had exactly such a setting; I wrote >> what I >> >>>>> wrote >> >>>>> because it is _still the wrong way around_ and as I happen to >> have added >> >>>>> to >> >>>>> unit tests this weekend _having suffered through precisely this >> >>>>> setting_. >> >>>>> >> >>>>> But we are on different wavelengths here and I evidently do not >> get my >> >>>>> point >> >>>>> across to you. And as you are the one who could make a change >> where it >> >>>>> matters, I have no choice but to rest my case in frustration. >> >>>> >> >>>> >> >>>> >> >>>> If you want to give up, then give up, but then don't complain >> about the >> >>>> current behaviour. If you want to fix it, then continue the >> discussion. >> >>>> >> >>>> You're right that we're on different wavelengths. If you want >> some tests >> >>>> to >> >>>> run at home but not on CRAN, then somewhere there has to be a >> >>>> conditional. >> >>>> I'm suggesting that the conditional should be "if there's a >> tight time >> >>>> limit, skip this". >> >>>> >> >>>> I don't remember if this was your suggestion, but someone has >> suggested >> >>>> "if >> >>>> we're running with the --as-cran option, skip this" and others >> have >> >>>> suggested "if we're running on CRAN, skip this". I don't see >> why you >> >>>> find >> >>>> my suggestion so objectionable. If you want, I'll repeat why I >> find the >> >>>> other two suggestions objectionable. >> >>> >> >>> >> >>> I agree with Duncan that having an option long/short makes more >> sense >> >>> than with/without cran, as long as cran sets that option to be >> short. >> >> >> >>> >> >>> >> >>> I would also prefer a command line switch to R CMD check to an >> >>> environment variable, but I'll be very happy with a standardized >> >>> environment variable. >> >> >> >> >> >> I honestly don't see the need for a standardized variable. I've >> told you >> >> how to detect that you are running --as-cran; if that isn't >> sufficient >> >> information, then you, the package author, need to set up >> something more >> >> elaborate, and assume that if it's not set up, then someone else >> (maybe >> >> CRAN) is running the test. >> >> > So maybe documenting that (_R_CHECK_TIMINGS_) more formally in >> R-exts >> > would be sufficient? >> >> > -Deepayan >> >> yes and no. >> As Duncan said very early, --as-cran is just turning on several >> already existing options, one of them being the >> _R_CHECK_TIMINGS_ -- which you can *also* enable by --timings. >> >> So, checking for _R_CHECK_TIMINGS_ is *not* checking for >> the presence of --as-cran ! >> .. in the sense that it is necessary but not sufficient. >> Also, if I run the "long checks" I may still want to use >> --timings. For me as package developer, the timings may be even more >> important in the "long" than in the "short" checks case. >> >> So, one solution that is little work would be that --as-cran >> sets an *extra* environment variable that is *only* set by >> --as-cran, but by no other command line switch. >> >> Still a pity that it seems people want to live in a 0/1 >> setting when it would be very natural to adopt a 0/1/2/3 >> (or so) one. >> It's a bit like prefering verbose = FALSE/TRUE as argument to >> an R function where eventually I often found that using a >> tracing = 0/1/2 (or then 'verbose' = 0/1/2 which is almost >> back compatible to FALSE/TRUE) >> was more useful. >> >> >> >> >> I asked the CRAN powers-that-be about the possibility of querying >> the amount >> >> of time remaining before a timeout; since the different platforms >> all use >> >> different mechanisms to enforce a timeout, that's not really >> practical. So >> >> the best you could hope for is to know that a timeout is in >> effect. Before >> >> I wrote any code, I'd need to hear why --as-cran detection isn't >> sufficient. >> >> I agree that *reliable* --as-cran detection solves the OP's >> and most other correspondents problem. >> But as argued above, _R_CHECK_TIMINGS_ is not sufficient. >> >> Martin > > > ______________________________________________ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel ______________________________________________ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
