2016-02-15 4:35 GMT+01:00 Sun, Rui <rui....@intel.com>: > Yes, JRI loads an R dynamic library into the executor JVM, which faces > thread-safe issue when there are multiple task threads within the executor. > > I am thinking if the demand like yours (calling R code in RDD > transformations) is much desired, we may consider refactoring RRDD for this > purpose, although it is currently intended for internal use by SparkR and not > a public API. So the RRDDs don't have that thread safety issue? I'm currently creating a new environment for each call, but it still crashes.
--------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
