I have just tried to pipe all the messages to the main thread – the new strategy is much easier to implement than I thought.
The new code works well. Indeed, as long as the Rprintf function is only called by the main thread, it doesn’t matter how many worker threads are created. From: Shian Su <s...@wehi.edu.au> Sent: Wednesday, 30 January 2019 2:47 PM To: Yang Liao <l...@wehi.edu.au> Cc: bioc-devel@r-project.org Subject: Re: [Untrusted Server]Re: [Bioc-devel] Rprintf in a multi-threaded environment Thanks Luke for clarifying that you should not call R API from outside the main thread ever. There’s some false advice in https://stackoverflow.com/questions/49723216/print-to-terminal-in-a-multithreaded-environment-for-cran-r-package that suggests it’d be ok if you set up the right mutex. I believe this is for C level parallelism so there’s no easy way to make it work with BiocParallel. You'd usually be able to slap a mutex over the prints so the shared output isn’t mangled. This approach works perfectly fine for printf() but can fail with Rprintf(), and we enforce the R API for output in package checks. On 30 Jan 2019, at 2:01 pm, Yang Liao <l...@wehi.edu.au<mailto:l...@wehi.edu.au>> wrote: Thanks Martin and Luke, I think the only way to go is to only use the main thread to handle the screen output. -----Original Message----- From: Tierney, Luke <luke-tier...@uiowa.edu<mailto:luke-tier...@uiowa.edu>> Sent: Wednesday, 30 January 2019 1:16 AM To: Yang Liao <l...@wehi.edu.au<mailto:l...@wehi.edu.au>> Cc: bioc-devel@r-project.org<mailto:bioc-devel@r-project.org> Subject: Re: [Bioc-devel] Rprintf in a multi-threaded environment No functions in the R API are safe to call from any thread other than the R main thread. -- Many may need to allocate from the R heap (more as ALTREP evaolves) and that is not thread safe; -- Many (and with some compilation options nearly all) can signal an error, and the subsequent jump can only work in the main thread. So: do not make R API calls from any thread other than the main thread. Ever. [Separate processes as created by approaches in the parallel package do not have these issues, though simple forking as in multicore can have other issues as pointed out by Martin Morgan.] Best, luke On Tue, 29 Jan 2019, Yang Liao wrote: Hi, I'm not sure if some C developers have gone through this problem: it seems that Rprintf cannot work safely in a multi-threaded environment. In particular, if I call Rprintf() from a then-created thread while the stack size checking is enabled (ie the "R_CStackLimit" pointer isn't set to -1), it is very likely to end up with some fatal error messages like: Error: C stack usage 847645293284 is too close to the limit Error: C stack usage 847336061668 is too close to the limit Error: C stack usage 847666277092 is too close to the limit Error: C stack usage 847346551524 is too close to the limit Error: C stack usage 847367531236 is too close to the limit Error: C stack usage 847357041380 is too close to the limit Error: C stack usage 847378021092 is too close to the limit Error: C stack usage 847655787236 is too close to the limit , and the R session terminates in a segfault. After I used all means to confirm that there was no memory leakage and the real stack use was minimum, I thought it can only be the Rprintf issue. I then disabled all screen outputs from the then-created threads and the error was gone. It was also reported on stackoverflow: https://stackoverflow.com/questions/50092949/why-does-rcout-and-rprint f-cause-stack-limit-error-when-multithreading I tried using a semaphore to protect all Rprintf calls but it didn't prevent the error. Since my program needs to report some messages from the worker threads (created by the main thread), I wonder if there is a solution to safely do so, or I have to pipe the messages to the main thread, which in turn calls Rprintf? I hope not to change "R_CStackLimit" to disable the stack size checks because it generates a "NOTE" in R check. Cheers, Yang _______________________________________________ The information in this email is confidential and\ > i...{{dropped:30}} _______________________________________________ Bioc-devel@r-project.org<mailto:Bioc-devel@r-project.org> mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel _______________________________________________ The information in this email is confidential and intended solely for the addressee. You must not disclose, forward, print or use it without the permission of the sender. The Walter and Eliza Hall Institute acknowledges the Wurundjeri people of the Kulin Nation as the traditional owners of the land where our campuses are located and the continuing connection to country and community. _______________________________________________ [[alternative HTML version deleted]] _______________________________________________ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
