On Mon, 3 Mar 2008, Brock Palen wrote:
I was writing a simple code to demonstrate deadlock. I create an
array of some number of doubles, when sizeof(double)*SIZE is >
btl_sm_eager_limit (on a smp machine) I expect the code to deadlock
but not until the buffer size is greater than the eager limit.
Now in 1.2.3 sm_eager_limit is 4096 (bytes?) So i expect a buffer of
doubles created with malloc to have up to 512 before the code
deadlocks. (remember this is for demonstrations only) The code
deadlocks at 504 doubles, does the eager limit not include the
envelope overhead? Is it really 64 Kbytes?
Thus does every message take up 64K no matter the size of buffer* ?
I think you missed an order of magnitude somewhere :). 504 doubles would
take 4032 bytes, so that would leave 64 bytes for header.
The eager limit is the total packet size, so headers are also included in
that space (explaining why you wouldn't be able to pack a full 512
doubles) into a single eager message.
There are two headers sent in the SM case - one for the PML (16 bytes) and
one for the SM BTL (16 bytes for 32 bit builds, 24 bytes for 64 bit
builds). So the grand total of header space for an SM message is 32 or 40
bytes, depending on build size. I'm not sure where the other 24 bytes
would be going in your case.
Hope this helps,
Brian
