On Nov 22, 2011, at 1:09 AM, Shamik Ganguly wrote: > I want to trace when the MPI library prevents an MPI_Send from going to the > socket and makes it access shared memory because the target node is on the > same chip (CMP). I want to use PIN to trace this. Can you please give me some > pointers about which functions are taking this decision so that I can > instrument the appropriate library calls in PIN?
What's PIN? The decision is made in the ob1 PML plugin. Way back during MPI_INIT, each MPI process creates lists of BTLs to use to contact each MPI process peer. When a process is on the same *node* (e.g., a single server) -- not just the same processor socket/chip -- the shared memory BTL is given preference to all other BTLs by use of a priority mechanism. Hence, the "sm" BTL is put at the front of the BML lists (BML = BTL multiplexing layer -- it's essentially just list management for BTLs). Later, when MPI_SEND comes through, it uses the already-setup BML lists to determine which BTL(s) to use to send a message. That's the 50,000 foot view. -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
