On 09/24/2011 08:55 AM, henry wrote:
Greetings,
I am trying to use Reference Class methods in Snowfall, using R 2.12.1
on Ubuntu Natty. Using then directly seems to work (stanza 2 below), but
using them indirectly does not (stanza 3 below). I get an "attempt to
apply non-function" error.
In addition to exporting the instance of the object I created to the
Snowfall slaves, I also made several attempts to export the method
itself (e.g. sfExport("instance$method") or some such), with no success.
Very strangely, if I run Stanza 2 below, then stanza 3 will work.
I tried emailing the Snowfall author, but have not heard back. Any help
is greatly appreciated!
Henry Bryant
Texas A&M University
library("snowfall")
library("methods")
# set up a simple reference class
calculator <- setRefClass("calculator")
calculator$methods(do_calc = function(x) {print(x*x)})
my_calc <- calculator$new()
wrapper <- function(x) {my_calc$do_calc(x)}
# STANZA 2: use snowfall without wrapper -- WORKING
#sfInit(parallel=TRUE, cpus=2, type="SOCK")
#sfExport("calculator")
#sfExport("my_calc")
#results <- sfLapply(1:10,my_calc$do_calc)
#sfStop()
#print(results)
# STANZA 3: use snowfall with wrapper -- NOT WORKING
# (not working by itself, but does work if the previous stanza is run
first!!)
sfInit(parallel=TRUE, cpus=2, type="SOCK")
sfExport("calculator")
sfExport("my_calc")
sfExport("wrapper")
last_results <- sfLapply(1:10,wrapper)
Hi Henry --
The problem here is that references classes implement their methods in a
kind of 'lazy' way -- my_calc$do_calc doesn't exist until it's actually
invoked for the first time, so when you
sfExport("my_calc")
you export the object but "do_calc" is defined as NULL
> sfClusterEval(my_calc$do_calc)
[[1]]
NULL
[[2]]
NULL
This seems to be a variant of the problem experienced elsewhere
https://stat.ethz.ch/pipermail/r-devel/2011-June/061260.html
http://stackoverflow.com/questions/7331518/method-initialisation-in-r-reference-classes
The solutions seem mostly ugly, e.g., to evaluate locally
> my_calc$do_calc
and then sfExport, or maybe a bit better as
calculator <- setRefClass("calculator")
calculator$methods(
initialize = function(...) {
callSuper(...)
.self$do_calc # force definition
.self
},
do_calc = function(x) x*x)
I'm unclear about whether there are other aspects of the reference class
implementation that tie the instance to the class definition in a way
that would undermine parallel evaluation.
Also in general it seems better to write truly 'functional programming'
style functions that don't rely on aspects of the environment to work,
e.g.,
wrapper <- function(x, calc) calc$do_calc(x)
sfLapply(1:10, wrapper, my_calc)
Martin
sfStop()
cat("\nRESULTS FROM SECOND TRY:\n")
print(last_results)
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Computational Biology
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N. PO Box 19024 Seattle, WA 98109
Location: M1-B861
Telephone: 206 667-2793
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
