Shaun Jackman wrote:
Eugene Loh wrote:
What's the rest? I said the shared-memory area is much smaller, but
I was confused about which OMPI release I was using. So, the
shared-memory area was 128 Mbyte and it was getting mapped in once
for each process, and so it was counted doubly.
If there are eight processes running on one host, does each process
allocate one 128 Mbyte shared memory buffer and map in the other seven
128 Mbyte buffers allocated by the other processes?
No. The total size for one, single shared file is computed and the
lowest rank on the node creates the file and mmaps it in. Then, the
other processes mmap the same file in.
The code is set up to have different "memory pools". E.g., look at
https://svn.open-mpi.org/source/xref/ompi-trunk/ompi/mca/btl/sm/btl_sm.c#sm_btl_first_time_init
. So, conceivably you could have different buffers on the same node.
But in practice it's just one buffer and, in any case, they're always
created by the lowest rank. E.g.,
https://svn.open-mpi.org/source/xref/ompi-trunk/ompi/mca/common/sm/common_sm_mmap.c#143
.
