My understanding about this issue with progressr is that it is nontrivial, and will take author Henrik Bengtsson some time & thought to resolve. OTOH if patches to 'parallel' were welcome, I could embed my solution in a modified version of parallel::mclapply by introducing 1 or 2 new parameters (e.g., mc.prog=NULL) to support progress reporting consistently with the existing interface.
From: Duncan Murdoch <murdoch.dun...@gmail.com> Date: Friday, July 23, 2021 at 11:26 AM To: Hugh Parsonage <hugh.parson...@gmail.com>, David Norris <da...@precisionmethods.guru> Cc: "r-package-devel@r-project.org" <r-package-devel@r-project.org> Subject: Re: [R-pkg-devel] Using parallel:::sendMaster &c. despite warnings On 23/07/2021 10:59 a.m., Hugh Parsonage wrote: Happily, package parallel is open-source. If you would like to make use of the unexported functions, you could copy the source code with acknowledgement into your package. There's a good chance that wouldn't work, because parallel is a base package. Base packages can work with R internals and user-written packages can't. Probably David's best course of action is to submit patches where necessary so that he doesn't need ::: access. The original motivation appeared to be avoiding inefficiency in a contributed package; if there's a patch that can fix that, it could be available in a few days. If it needs changes to the parallel package (e.g. exposing functions that are currently internal), that will take much longer, and needs a strong argument why the current API isn't sufficient. Alternatively, he can go ahead and use :::, but just not expect his package to be on CRAN. There are other ways to distribute packages. Duncan Murdoch Naturally this might be a lot of work and risks different behaviour between your package and parallel, but neither might be that terrible for you. And certainly would be better than trying to pretend that the functions you want are exported from parallel. On Fri, 23 Jul 2021 at 23:09, David Norris <mailto:da...@precisionmethods.guru> wrote: I can confirm that getFromNamespace() does indeed circumvent the WARNING. I might otherwise prefer ':::', however, for making its 'bad intentions' so clear. I was very much aware of the cautionary language under `?readChild`, but wondered whether *in practice* more liberal community policies might be in force pending the maturation of the futureverse. Although the sparse history of the relevant file hardly supports "might-break-at-any-time" alarmism, I do note that the last commit (in May 2020) introduced an argument sendMaster(raw.asis=TRUE) that I am actually using. https://github.com/wch/r-source/commits/trunk/src/library/parallel/R/unix/mcfork.R (Also agree with Yohann that ideally one might support Windows with a warning such as data.table issues regarding my Mac's non-support for OpenMP.) Best, David From: Sebastian Meyer <mailto:seb.me...@fau.de> Organization: Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) Date: Friday, July 23, 2021 at 8:05 AM To: David Norris <mailto:da...@precisionmethods.guru>, "mailto:r-package-devel@r-project.org"; <mailto:r-package-devel@r-project.org> Subject: Re: [R-pkg-devel] Using parallel:::sendMaster &c. despite warnings Am 23.07.21 um 13:19 schrieb David Norris: Because parallelized progress reporting in the futureverse.org incurs latencies too great for my application (https://github.com/HenrikBengtsson/progressr/issues/118), I have found it necessary to implement my own progress reporting using some of the non-exported functionality from `parallel`. (I do appreciate that Windows lacks the fork() system call, and will not support this. But am willing to make this an OS_type: unix-only package.) Of course, I get a WARNING for this: ── R CMD check results ──────────────────────────────── precautionary 0.2.6 ──── Duration: 6m 41.8s ❯ checking dependencies in R code ... WARNING Unexported objects imported by ':::' calls: ‘parallel:::readChild’ ‘parallel:::selectChildren’ ‘parallel:::sendMaster’ See the note in ?`:::` about the use of this operator. Including base/recommended package(s): ‘parallel’ Is this warning an absolute deal-killer on CRAN? Is there a 'correct' way to do `:::` that avoids the warning altogether? The 'parallel' functions your package intends to access seem to be intentionally unexported. Their help page says: "They are not available on Windows, and not exported from the namespace", and "This is a very low-level interface for expert use only: it not regarded as part of the R API and subject to change without notice." Correspondingly, the CRAN Repository Policy says Also, ::: should not be used to access undocumented/internal objects in base packages (nor should other means of access be employed). Such usages can cause packages to break at any time, even in patched versions of R. which kind of answers both of your questions. The policy thus implicitly advises against using getFromNamespace(). Best regards, Sebastian Meyer Kind regards, David Norris ______________________________________________ mailto:R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel ______________________________________________ mailto:R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel ______________________________________________ mailto:R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel ______________________________________________ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
