Re: [OMPI users] multiple LIDs

2006-12-06 Thread Jeff Squyres
On Dec 6, 2006, at 2:05 AM, Gleb Natapov wrote: Actually I was wondering why there is a facility for having multiple LIDs for the same port. This led me to the entire series of questions. It is still not very clear to, as to what is the advantage of assigning multiple LIDs to the same p

Re: [OMPI users] multiple LIDs

2006-12-06 Thread Gleb Natapov
On Wed, Dec 06, 2006 at 12:14:35PM +0530, Chevchenkovic Chevchenkovic wrote: > Hi, > Actually I was wondering why there is a facility for having multiple > LIDs for the same port. This led me to the entire series of questions. >It is still not very clear to, as to what is the advantage of > a

Re: [OMPI users] multiple LIDs

2006-12-06 Thread Chevchenkovic Chevchenkovic
Hi, Actually I was wondering why there is a facility for having multiple LIDs for the same port. This led me to the entire series of questions. It is still not very clear to, as to what is the advantage of assigning multiple LIDs to the same port. Does it give some performance advantages? -Chev

Re: [OMPI users] multiple LIDs

2006-12-04 Thread Jeff Squyres
There are two distinct layers of software being discussed here: - the PML (basically the back-end to MPI_SEND and friends) - the BTL (byte transfer layer, the back-end bit movers for the ob1 and dr PMLs -- this distinction is important because there is nothing in the PML design that forces th

Re: [OMPI users] multiple LIDs

2006-12-04 Thread Gleb Natapov
On Mon, Dec 04, 2006 at 11:57:07PM +0530, Chevchenkovic Chevchenkovic wrote: > Thanks for that. > > Suppose, if there there are multiple interconnects, say ethernet and > infiniband and a million byte of data is to be sent, then in this > case the data will be sent through infiniband (since it

Re: [OMPI users] multiple LIDs

2006-12-04 Thread Chevchenkovic Chevchenkovic
Thanks for that. Suppose, if there there are multiple interconnects, say ethernet and infiniband and a million byte of data is to be sent, then in this case the data will be sent through infiniband (since its a fast path .. please correct me here if i m wrong). If there are mulitple such send

Re: [OMPI users] multiple LIDs

2006-12-04 Thread Gleb Natapov
On Mon, Dec 04, 2006 at 10:53:26PM +0530, Chevchenkovic Chevchenkovic wrote: > Hi, > It is not clear from the code as mentioned by you from > ompi/mca/pml/ob1/ where exactly the selection of BTL bound to a > particular LID occurs. Could you please specify the file/function name > for the same? Th

Re: [OMPI users] multiple LIDs

2006-12-04 Thread Chevchenkovic Chevchenkovic
Hi, It is not clear from the code as mentioned by you from ompi/mca/pml/ob1/ where exactly the selection of BTL bound to a particular LID occurs. Could you please specify the file/function name for the same? -chev On 12/4/06, Gleb Natapov wrote: On Mon, Dec 04, 2006 at 01:07:08AM +0530, Chevc

Re: [OMPI users] multiple LIDs

2006-12-04 Thread Gleb Natapov
On Mon, Dec 04, 2006 at 01:07:08AM +0530, Chevchenkovic Chevchenkovic wrote: > Also could you please tell me which part of the openMPI code needs to > be touched so that I can do some modifications in it to incorporate > changes regarding LID selection... > It depend what do you want to do. The pa

Re: [OMPI users] multiple LIDs

2006-12-04 Thread Gleb Natapov
On Mon, Dec 04, 2006 at 01:02:48AM +0530, Chevchenkovic Chevchenkovic wrote: > Is it possible to control the LID where the send and recvs are > posted.. on either ends? No, but you can control how much LIDs will be used from available LIDs. This can be configured with "btl_openib_max_lmc" parameter

Re: [OMPI users] multiple LIDs

2006-12-03 Thread Chevchenkovic Chevchenkovic
Also could you please tell me which part of the openMPI code needs to be touched so that I can do some modifications in it to incorporate changes regarding LID selection... On 12/4/06, Chevchenkovic Chevchenkovic wrote: Is it possible to control the LID where the send and recvs are posted.. on

Re: [OMPI users] multiple LIDs

2006-12-03 Thread Chevchenkovic Chevchenkovic
Is it possible to control the LID where the send and recvs are posted.. on either ends? On 12/3/06, Gleb Natapov wrote: On Sun, Dec 03, 2006 at 07:03:33PM +0530, Chevchenkovic Chevchenkovic wrote: > Hi, > I had this query. I hope some expert replies to it. > I have 2 nodes connected point-to-p

Re: [OMPI users] multiple LIDs

2006-12-03 Thread Gleb Natapov
On Sun, Dec 03, 2006 at 07:03:33PM +0530, Chevchenkovic Chevchenkovic wrote: > Hi, > I had this query. I hope some expert replies to it. > I have 2 nodes connected point-to-point using infiniband cable. There > are multiple LIDs for each of the end node ports. >When I give an MPI_Send, Are the

[OMPI users] multiple LIDs

2006-12-03 Thread Chevchenkovic Chevchenkovic
Hi, I had this query. I hope some expert replies to it. I have 2 nodes connected point-to-point using infiniband cable. There are multiple LIDs for each of the end node ports. When I give an MPI_Send, Are the sends are posted on different LIDs on each of the end nodes OR they are they posted on