On Fri, Feb 14, 2014 at 3:43 AM, Jakub Jelinek <ja...@redhat.com> wrote: > On Fri, Feb 14, 2014 at 09:21:24AM +0100, Jakub Jelinek wrote: >> Well, fork is async signal safe function, so calling malloc/free, or any >> kind of synchronization primitives is completely unsafe there. >> >> The only safe thing could be to atomically or in some global flag (or set >> some TLS flag?) and deal with the freeing next time you encounter omp >> parallel. But, the state of the old thread pool may be in some inconsistent >> shape. > > BTW, I think far cleaner solution would be to discuss on Omp-lang and add > some standard omp_* function which would allow to throw away all the cached > OpenMP threads, after calling that function one could not assume > threadprivate vars (other than in the initial thread) preserve their values. > If this function would be only allowed outside of the parallel region (i.e. > if omp_in_parallel () == 0, or even just if omp_get_level () == 0) and > pretend to do > #pragma omp parallel num_threads (1) > ; > i.e. something after which it isn't guaranteed to preserve threadprivate > vars, then the library could perform this at the point where it is safe to > do so (of course it wouldn't be async-signal-safe function) and isn't a > performance issue (calling it when you are expecting to soon launch another > #pragma omp parallel could of course slow things down a lot). > > Anything else is going to be either unsafe, or leak memory.
I think the core problem here is that it's not possible to "hide" OMP usage inside an interface boundary, so you can't reliably compose larger programs out of pieces that individually may or may not use OMP. And unfortunately, I don't think your proposed omp_forget_threadprivates() function helps solve that problem. Like, consider the case of DGEMM implemented using OMP. Right now, this means that calling DGEMM will break fork(). If DGEMM instead called omp_forget_threadprivates(), then fork() would work. BUT, the program might contain other code -- perhaps in a different library -- which also used threadprivates. And every time someone calls DGEMM, this other code would find that their threadprivates had mysteriously disappeared. (Or might find this, depending on whether fork() was called in between, etc.) Whether it would be safe to call omp_forget_threadprivates() is a global property of the whole program, not something that can be determined by looking at any one piece in isolation. What *would* work is if GOMP started tracking whether there were any threadprivate variables in existence. Then it would be possible to dynamically determine at runtime whether this global property held, and automatically clean up threads at pre-fork-time iff it was globally safe to do so. Then we could document that *if* you want to use OMP inside a composable library, then you have to avoid threadprivates, and everything would just work. (In this particular case, it is true that OpenBLAS already doesn't use any threadprivates...) -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org
