Also I have no idea what the memory window question is, i will
look it up on google.
aon075:~ root# dmesg | grep GM
GM: gm_register_memory will be able to lock 96000 pages (375 MBytes)
This just answered it - there is 375MB available for GM to register, which
is the IOMMU window size available to the GM driver. This is quite small,
and it would be very helpful if this could be increased. helpful == much
better performance for your jobs.
It's possible that OMPI is not managing registered space well, as is
required when the aggregate registered memory needed by a job is larger than
the memory available to be registered. I do some research locally into
OMPIs management of registered memory when using GM. If it comes to it,
would you be willing to run an OMPI with some debug statements in it for me?
thanks,
-reese
