Hi,
Thanks for reporting the bug. Now fixed in 1.13.4 (devel) and 1.2.2
(release).
> bplapply(1:2, print, BPPARAM=SnowParam(outfile = NULL))
starting worker for localhost:11031
starting worker for localhost:11031
Type: EXEC
[1] 1
Type: EXEC
[1] 2
Type: DONE
Type: DONE
[[1]]
[1] 1
[[2]]
[1] 2
Some new features have been added to BiocParallel - logging with
futile.logger, writing out log and result files and control over how a
job is divided into tasks.
## send log file to 'logdir':
bplapply(1:2, print, BPPARAM=SnowParam(log=TRUE, logdir=tempdir())
## write results to 'resdir':
bplapply(1:2, print, BPPARAM=SnowParam(resdir=tempdir())
## by default jobs are divided evenly over the workers
bplapply(1:100, print, BPPARAM=SnowParam(workers=4))
## force the job to be run in 2 tasks
## (useful when you know runtime or memory requirements)
bplapply(1:100, print, BPPARAM=SnowParam(tasks=2))
It would be great if you had a chance to try any of these out - I'd be
interested in the feedback. Logging was intended to take the idea of the
'outfile' from snow further with the ability to add messages that can be
filtered by threshold.
The package also has a new errors/logging vignette:
http://www.bioconductor.org/packages/3.2/bioc/vignettes/BiocParallel/inst/doc/Errors_Logs_And_Debugging.pdf
Valerie
On 05/21/2015 07:58 AM, Leonardo Collado Torres wrote:
Hi,
This might be a BioC support website question, but maybe it's a bug.
In previous versions of BiocParallel, you could specify where to
direct the output from the workers by using the 'outfile' argument.
For example, SnowParam(outfile = Sys.getenv('SGE_STDERR_PATH')). I'm
not finding how to do so with the current version (1.3.12).
I understand that SnowParam() has a ... argument that gets passed to
snow::makeCluster(), according to the SnowParam() docs. I also see
that when log = TRUE, a script is used instead of snow. But log =
FALSE by default.
If I use snow, the 'outfile' argument does work as shown below:
## Nothing gets printed by default
cl <- makeCluster(2, type = 'SOCK')
y <- clusterApply(cl, 1:2, print)
## Use outfile now, print works
cl <- makeCluster(2, type = 'SOCK', outfile = NULL)
starting worker for localhost:11671
starting worker for localhost:11671
y <- clusterApply(cl, 1:2, print)
Type: EXEC
Type: EXEC
[1] 1
[1] 2
packageVersion('snow')
[1] ‘0.3.13’
However, I can't use 'outfile' with SnowParam:
## SerialParam works
x <- bplapply(1:2, print, BPPARAM = SerialParam())
[1] 1
[1] 2
## No printing by default with SnowParam, as expected
x <- bplapply(1:2, print, BPPARAM = SnowParam(workers = 1))
## Can't pass 'outfile' argument
x <- bplapply(1:2, print, BPPARAM = SnowParam(workers = 1, outfile = NULL))
Error in bplapply(1:2, print, BPPARAM = SnowParam(workers = 1, outfile
= NULL)) :
error in evaluating the argument 'BPPARAM' in selecting a method for
function 'bplapply': Error in envRefSetField(.Object, field, classDef,
selfEnv, elements[[field]]) :
‘outfile’ is not a field in class “SnowParam”
Digging at the code, I see that in SnowParam() the ... argument is
saved in .clusterargs
args <- c(list(spec = workers, type = type), list(...))
.clusterargs <- lapply(args, force)
However, ... is still passed to .SnowParam(), and .SnowParam() fields are:
fields=list(
cluster="cluster",
.clusterargs="list",
.controlled="logical",
log="logical",
threshold="ANY",
logdir="character",
resultdir="character")
So, I'm wondering if ... should not be passed to .SnowParam().
## Trying to pass outfile to .clusterargs directly doesn't work
## Actually, I'm surprised it didn't crash .SnowParam()
x <- bplapply(1:2, print, BPPARAM = SnowParam(workers = 1,
.clusterargs=list(outfile = NULL)))
Full log: https://gist.github.com/9ac957c6cad1c07f4ea4
Thanks,
Leo
_______________________________________________
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
--
Computational Biology / Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, Seattle, WA 98109
Email: voben...@fredhutch.org
Phone: (206) 667-3158
_______________________________________________
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
