Hi Yan, Sorry it wasn't the simple answer. On my machine, your code creates 3 R processes (#cores -1) and they are all active.
If you have an even simpler example say parLapply(cl, 1:8, function_that_takes_a_while) does that get all the cores going? John On Fri, Dec 11, 2015 at 6:58 AM, ALPEROVYCH Yan <alperov...@em-lyon.com> wrote: > Hi John, > > Thank you for the reply. In fact my original list has 122^2 elements in it > (I simplified the code here), and parLapply behaves in a similar way still. > Can it be something else? > > Yan > > On Dec 11, 2015, at 1:51 PM, John Magnotti <john.magno...@gmail.com > <mailto:john.magno...@gmail.com>> wrote: > > Hello Yan, > > I think parLapply is just assigning a core for every item in the list, G, > you supplied. Because you have more cores than items in the list, some of > the cores won't receive any work. > > > John > > On Fri, Dec 11, 2015 at 4:00 AM, ALPEROVYCH Yan <alperov...@em-lyon.com > <mailto:alperov...@em-lyon.com>> wrote: > Hello, > > I have a piece of code that needs parallelization and it used to work just > fine before (about 6 months ago). However, I had to rerun it yesterday and > found out that my code is now behaving in a weird way - not all worker > processes are charged with the computation. I created a little code that > allows reproducing the issue. Here is a snapshot of the top command: > > PID COMMAND %CPU TIME > 872 R 97.7 00:02.45 > 871 R 0.0 00:00.03 > 870 R 98.2 00:02.93 > 869 R 0.0 00:00.03 > 868 R 97.7 00:02.46 > 867 R 0.0 00:00.03 > 866 R 94.4 00:02.36 > 862 R 1.0 00:04.64 > > Interestingly, the mclapply command seems to correctly charge all workers > instantly. > > So is this an intended behavior? > > Example that reproduces the issue on my machine: > # > rm(list = ls()) > library(parallel) > set.seed(123) > # > expr <- expression(pnorm(b0 + b1*x1 + b2*x2 + b3*x3)) > G <- list( > D(D(expr, "x2"), "b0"), > D(D(expr, "x2"), "b1"), > D(D(expr, "x2"), "b2"), > D(D(expr, "x2"), "b3")) > # > b0 <- 1.2 > b1 <- 0.4 > b2 <- 0.2 > b3 <- -0.6 > x1 <- rnorm(10^7) > x2 <- rnorm(10^7) > x3 <- rnorm(10^7) > # > nc <- detectCores() - 1 > cl <- makeCluster(nc, type = "FORK") > grad_g <- parLapply(cl, G, function(Z) lapply(Z, function(x) > mean(eval(x)))) > stopCluster(cl) > # > sessionInfo() > R version 3.2.3 (2015-12-10) > Platform: x86_64-apple-darwin13.4.0 (64-bit) > Running under: OS X 10.11.2 (El Capitan) > > locale: > [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8 > > attached base packages: > [1] parallel stats graphics grDevices utils datasets methods > base > > > > > ---- > Ce message electronique et tous les fichiers attaches qu'il contient sont > confidentiels et destines exclusivement à l'usage de la personne à laquelle > ils sont adresses. Si vous avez reçu ce message par erreur, merci de le > retourner à son metteur. Les idees et opinions presentees dans ce message > sont celles de son auteur, et ne representent pas necessairement celles de > l'institution ou entite affiliee dont l'auteur est l'employe. La > publication, l'usage, la distribution, l'impression ou la copie non > autorisee de ce message et des attachements qu'il contient sont strictement > interdits. > > This email and any files transmitted with it are confide...{{dropped:10}} > > _______________________________________________ > R-SIG-Mac mailing list > R-SIG-Mac@r-project.org<mailto:R-SIG-Mac@r-project.org> > https://stat.ethz.ch/mailman/listinfo/r-sig-mac > > > ---- > Ce message electronique et tous les fichiers attaches qu'il contient sont > confidentiels et destines exclusivement à l'usage de la personne à laquelle > ils sont adresses. Si vous avez reçu ce message par erreur, merci de le > retourner à son metteur. Les idees et opinions presentees dans ce message > sont celles de son auteur, et ne representent pas necessairement celles de > l'institution ou entite affiliee dont l'auteur est l'employe. La > publication, l'usage, la distribution, l'impression ou la copie non > autorisee de ce message et des attachements qu'il contient sont strictement > interdits. > > This email and any files transmitted with it are confi...{{dropped:15}} _______________________________________________ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac