On Jun 20, 2016, at 4:27 PM, Alex A. Granovsky <g...@classic.chem.msu.su> wrote: > > Would be the use of mlockall helpful for this approach?
That's an interesting idea; I didn't know about the existence of mlockall(MCL_FUTURE). It has a few drawbacks, of course (e.g., processes can't shrink), but they might be acceptable tradeoffs for typical HPC/MPI application scenarios. Keep in mind that locking memory is only one half of the registration process: the other half is notifying the network stack so that they can record/track virtual memory-->physical memory mapping. Meaning: Open MPI will still need its registration cache infrastructure -- but it *might* be able to be slightly simpler because the eviction mechanisms will never be invoked. Also keep in mind that the memory mechanisms -- regardless of whether it's mlockall(MCL_FUTURE) or the newly-revamped "patcher" system in the upcoming Open MPI v2.0.0 -- are not in the critical performance path (e.g., that stuff doesn't happen during a call to MPI_SEND). The part that *is* in the critical performance path is the registration cache -- i.e., the part where Open MPI asks "is this buffer/memory registered [with the network stack]?" That part is designed to be fast and, at least at the moment, will still need to be there. If there's some kind of equivalent to mlockall(MCL_FUTURE) that *also* transparently registers all new memory with the relevant underlying network stack(s) and contexts, that would be neat. -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
