Ben,
You may try to disable registration cache, it may relieve pressure on memory
resources.
--mca mpi_leave_pinned 0
You may find a bit more details here:
http://www.open-mpi.org/faq/?category=openfabrics#large-message-leave-pinned
Using the option you may observe drop in BW performance.
R
Hi,
I would suggest use MXM (part of mofed, can be downloaded as standalone rpm
from http://mellanox.com/products/mxm for ofed)
It uses UD (constant memory footprint) and should provide good performance.
The next MXM v2.0 will support RC and DC (reliable UD) as well.
Once mxm is installed from rp