Re: [OMPI users] RoCE (IBoE) & OpenMPI
As far as I remember we don't allow to user to specify SL for RoCE. RoCE considered kinda ethernet device and RDMACM connection manager is used to setup the connections. it means that in order to select network X or Y, you may use ip/netmask (btl_openib_ipaddr_include) . Pavel (Pasha) Shamis --- Application Performance Tools Group Computer Science and Math Division Oak Ridge National Laboratory On Feb 18, 2011, at 4:14 PM, Michael Shuey wrote: > Per-node GID & SL settings == bad. Site-wide GID & SL settings == good. > > If this could be an MCA param (like btl_openib_ib_service_level) > that'd be great - we already have a global config file of similar > params. We'd definitely want the same N everywhere. > > -- > Mike Shuey > > > > On Fri, Feb 18, 2011 at 3:44 PM, Jeff Squyres wrote: >> On Feb 18, 2011, at 1:39 PM, Michael Shuey wrote: >> >>> RoCE HCAs keep a GID table, like normal HCAs. Every time you bring up >>> a vlan interface, another entry gets automatically added to the table. >>> If I select one of these other GIDs, packets get a VLAN tag, and that >>> contains the necessary priority bits (well, assuming I selected the >>> right IB service level, which is mapped to the priority tag in the >>> VLAN header) for the traffic to match a lossless class of service on >>> the switch. >> >> Ah -- I see it now (it's been a looong time since I've looked in Open MPI's >> verbs code!). We query and simply take the 0th GID from a given IBV device >> port's GID table. >> >>> For this to work, I really need for the IB client to select a >>> non-default GID. A few test programs included in OFED will do this, >>> but I'm not sure OpenMPI will. Any thoughts? >> >> Yes, we can do this. It's pretty easy to add an MCA parameter to select the >> Nth GID rather than always taking the 0th. >> >> To make this simple, can you make it so that the value of N is the same >> across all nodes in your cluster? Then you can set a site-wide MCA param >> for that value of N and be done with this issue. If we have to have a >> per-node setting of N, it could get a little hairy (it's do-able, but... >> it's a heckuva lot easier if N is the same everywhere). >> >> -- >> Jeff Squyres >> jsquy...@cisco.com >> For corporate legal information go to: >> http://www.cisco.com/web/about/doing_business/legal/cri/ >> >> > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
[OMPI users] Selecting different processors during function
Hi everybody,
I'm trying to write a routine that, if a certain condition is met, it
transfers a specific string from processor A to processor B. However, during
this transfer, I have to access the processor B within the same function
(the function was accessed on A, but I need to go to B to put the
transferred string on it's right place). The problem is that I have no idea
how to change between processors while using one of them. Here's a sample of
the code:
for ( int j = 0; j < atom->nlocal; j++ )
{
if ( rank != atom->tag[j] ) //condition for transfer
{
dest = atom->tag[j]; //the one who needs to deliver to
receive
atom->tag[j] = rank;
MPI_Isend( noisez[j], noisesize, MPI_DOUBLE, rank, 0, world,
&request);
}
}
/* I know this doesn't work, but it's just to get the idea of what I'm
trying to do */
if( rank == dest ) //trying to access the destination processor
{
for ( int n = 0; n < atom->nlocal; n++ )
{
if( n =! atomtag[n] )
{
MPI_Irecv( noisez[n], noisesize, MPI_DOUBLE, flag2, 0,
world, &request);
MPI_Wait(&request,&stat);
for ( int m = n; m < atom->nlocal; m++ )
atomtag[m] = m;
break;
}
else if ( n == atom->nlocal -1)
{
MPI_Irecv( noisez[atom->nlocal], noisesize, MPI_DOUBLE,
flag2, 0, world, &request);
MPI_Wait(&request,&stat);
}
}
}
Is there any function to call another processor while using one of them?
Regards,
Alexandre
Re: [OMPI users] Selecting different processors during function
Your question actually doesn't make sense in an MPI application. In MPI, you
would have two independent processes running. One does the send, and the other
does the receive. Both processes are running all the time, each on its own
processor.
So you don't "switch" to another processor - the receiving process simply sits
waiting for the message to arrive from the sender, and then proceeds to do the
desired actions.
On Feb 19, 2011, at 9:37 AM, Alexandre Barrozo wrote:
> Hi everybody,
>
> I'm trying to write a routine that, if a certain condition is met, it
> transfers a specific string from processor A to processor B. However, during
> this transfer, I have to access the processor B within the same function
> (the function was accessed on A, but I need to go to B to put the transferred
> string on it's right place). The problem is that I have no idea how to change
> between processors while using one of them. Here's a sample of the code:
>
> for ( int j = 0; j < atom->nlocal; j++ )
> {
> if ( rank != atom->tag[j] ) //condition for transfer
> {
> dest = atom->tag[j]; //the one who needs to deliver to
> receive
>
> atom->tag[j] = rank;
>
> MPI_Isend( noisez[j], noisesize, MPI_DOUBLE, rank, 0, world,
> &request);
> }
> }
>
> /* I know this doesn't work, but it's just to get the idea of what I'm
> trying to do */
>
> if( rank == dest ) //trying to access the destination processor
> {
> for ( int n = 0; n < atom->nlocal; n++ )
> {
> if( n =! atomtag[n] )
> {
>
> MPI_Irecv( noisez[n], noisesize, MPI_DOUBLE, flag2, 0, world,
> &request);
> MPI_Wait(&request,&stat);
> for ( int m = n; m < atom->nlocal; m++ )
> atomtag[m] = m;
> break;
> }
> else if ( n == atom->nlocal -1)
> {
>
> MPI_Irecv( noisez[atom->nlocal], noisesize, MPI_DOUBLE,
> flag2, 0, world, &request);
> MPI_Wait(&request,&stat);
> }
> }
> }
>
> Is there any function to call another processor while using one of them?
>
> Regards,
>
> Alexandre
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
