[OMPI users] Open MPI data transfer error

[Jack Bryan]

Hi, 
In my Open MPI program, one master sends data to 3 workers.
Two workers can receive their data. 
But, the third  worker can not get their data. 
Before sending data, the master sends a head information to each worker 
receiver so that each worker knows what the following data package is. (such as 
length, package tag). The third worker can get its head information message 
from master but cannot get its correct data package. 
It got the data that should be received by first worker, which get its correct 
data. 
Why ? 
Any help is appreciated. 
thanks
Jack
Nov. 4 2010

  

Re: [OMPI users] Open MPI data transfer error

[Prentice Bisbal]

Jack Bryan wrote:
> 
> Hi, 
> 
> In my Open MPI program, one master sends data to 3 workers.
> 
> Two workers can receive their data. 
> 
> But, the third  worker can not get their data. 
> 
> Before sending data, the master sends a head information to each worker
> receiver 
> so that each worker knows what the following data package is. (such as
> length, package tag).
>  
> The third worker can get its head information message from master but
> cannot get its correct 
> data package. 
> 
> It got the data that should be received by first worker, which get its
> correct data. 
> 


Jack,

Providing the relevant sections of code here would be very helpful.


I would tell you to add some printf statements to your code to see what
data is stored in your variables on the master before it sends them to
each node, but Jeff Squyres and I agreed to disagree in a civil manner
on that debugging technique earlier this week, and I'd hate to re-open
those old wounds by suggesting that technique here. ;)


-- 
Prentice

Re: [OMPI users] Open MPI data transfer error

[Jack Bryan]

Thanks, I have used "cout" in c++ to print the values of data. 
The sender sends correct data to correct receiver. 
But, receiver gets wrong data from correct sender. 
why ? 
thanks 
Nov. 5 2010
> Date: Fri, 5 Nov 2010 08:54:22 -0400
> From: prent...@ias.edu
> To: us...@open-mpi.org
> Subject: Re: [OMPI users] Open MPI data transfer error
> 
> Jack Bryan wrote:
> > 
> > Hi, 
> > 
> > In my Open MPI program, one master sends data to 3 workers.
> > 
> > Two workers can receive their data. 
> > 
> > But, the third  worker can not get their data. 
> > 
> > Before sending data, the master sends a head information to each worker
> > receiver 
> > so that each worker knows what the following data package is. (such as
> > length, package tag).
> >  
> > The third worker can get its head information message from master but
> > cannot get its correct 
> > data package. 
> > 
> > It got the data that should be received by first worker, which get its
> > correct data. 
> > 
> 
> 
> Jack,
> 
> Providing the relevant sections of code here would be very helpful.
> 
> 
> I would tell you to add some printf statements to your code to see what
> data is stored in your variables on the master before it sends them to
> each node, but Jeff Squyres and I agreed to disagree in a civil manner
> on that debugging technique earlier this week, and I'd hate to re-open
> those old wounds by suggesting that technique here. ;)
> 
> 
> -- 
> Prentice
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
  

Re: [OMPI users] Open MPI data transfer error

[Prentice Bisbal]

We can't help you with your coding problem without seeing your code.


Jack Bryan wrote:
> Thanks, 
> I have used "cout" in c++ to print the values of data. 
> 
> The sender sends correct data to correct receiver. 
> 
> But, receiver gets wrong data from correct sender. 
> 
> why ? 
> 
> thanks 
> 
> Nov. 5 2010
> 
>> Date: Fri, 5 Nov 2010 08:54:22 -0400
>> From: prent...@ias.edu
>> To: us...@open-mpi.org
>> Subject: Re: [OMPI users] Open MPI data transfer error
>>
>> Jack Bryan wrote:
>> >
>> > Hi,
>> >
>> > In my Open MPI program, one master sends data to 3 workers.
>> >
>> > Two workers can receive their data.
>> >
>> > But, the third worker can not get their data.
>> >
>> > Before sending data, the master sends a head information to each worker
>> > receiver
>> > so that each worker knows what the following data package is. (such as
>> > length, package tag).
>> >
>> > The third worker can get its head information message from master but
>> > cannot get its correct
>> > data package.
>> >
>> > It got the data that should be received by first worker, which get its
>> > correct data.
>> >
>>
>>
>> Jack,
>>
>> Providing the relevant sections of code here would be very helpful.
>>
>> 
>> I would tell you to add some printf statements to your code to see what
>> data is stored in your variables on the master before it sends them to
>> each node, but Jeff Squyres and I agreed to disagree in a civil manner
>> on that debugging technique earlier this week, and I'd hate to re-open
>> those old wounds by suggesting that technique here. ;)
>> 
>>
>> --
>> Prentice

Re: [OMPI users] Open MPI data transfer error

[David Zhang]

As Prentice said, we can't help you without seeing your code.  openMPI has
stood many trials from many programmers, with many bugs ironed out. So
typically it is unlikely openMPI is the source of your error.  Without
seeing your code the only logical conclusion is that something is wrong with
your programming.

On Fri, Nov 5, 2010 at 10:52 AM, Prentice Bisbal  wrote:

> We can't help you with your coding problem without seeing your code.
>
>
> Jack Bryan wrote:
> > Thanks,
> > I have used "cout" in c++ to print the values of data.
> >
> > The sender sends correct data to correct receiver.
> >
> > But, receiver gets wrong data from correct sender.
> >
> > why ?
> >
> > thanks
> >
> > Nov. 5 2010
> >
> >> Date: Fri, 5 Nov 2010 08:54:22 -0400
> >> From: prent...@ias.edu
> >> To: us...@open-mpi.org
> >> Subject: Re: [OMPI users] Open MPI data transfer error
> >>
> >> Jack Bryan wrote:
> >> >
> >> > Hi,
> >> >
> >> > In my Open MPI program, one master sends data to 3 workers.
> >> >
> >> > Two workers can receive their data.
> >> >
> >> > But, the third worker can not get their data.
> >> >
> >> > Before sending data, the master sends a head information to each
> worker
> >> > receiver
> >> > so that each worker knows what the following data package is. (such as
> >> > length, package tag).
> >> >
> >> > The third worker can get its head information message from master but
> >> > cannot get its correct
> >> > data package.
> >> >
> >> > It got the data that should be received by first worker, which get its
> >> > correct data.
> >> >
> >>
> >>
> >> Jack,
> >>
> >> Providing the relevant sections of code here would be very helpful.
> >>
> >> 
> >> I would tell you to add some printf statements to your code to see what
> >> data is stored in your variables on the master before it sends them to
> >> each node, but Jeff Squyres and I agreed to disagree in a civil manner
> >> on that debugging technique earlier this week, and I'd hate to re-open
> >> those old wounds by suggesting that technique here. ;)
> >> 
> >>
> >> --
> >> Prentice
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>



-- 
David Zhang
University of California, San Diego

[OMPI users] Deprecated parameter: plm_rsh_agent

[Joshua Bernstein]

Hello All,

When building the examples included with OpenMPI version 1.5 I see a 
message printed as follows:


--
A deprecated MCA parameter value was specified in an MCA parameter
file.  Deprecated MCA parameters should be avoided; they may disappear
in future releases.

  Deprecated parameter: plm_rsh_agent
--

While I know that in pre 1.3.x releases the variable was pls_rsh_agent, 
plm_rsh_agent worked all the way through at least 1.4.3. What is the new 
keyword name? I can't seem to find it in the FAQ located here:


http://www.open-mpi.org/faq/?category=rsh

-Josh

Re: [OMPI users] Deprecated parameter: plm_rsh_agent

[Samuel K. Gutierrez]

Hi Josh,

I -think- the new name is orte_rsh_agent.  At least according to  
ompi_info.


$ ompi_info -a --parsable | grep orte_rsh_agent
mca:orte:base:param:orte_rsh_agent:value:ssh : rsh
mca:orte:base:param:orte_rsh_agent:data_source:default value
mca:orte:base:param:orte_rsh_agent:status:writable
mca:orte:base:param:orte_rsh_agent:help:The command used to launch  
executables on remote nodes (typically either "ssh" or "rsh")

mca:orte:base:param:orte_rsh_agent:deprecated:no
mca:orte:base:param:orte_rsh_agent:synonym:name:pls_rsh_agent
mca:orte:base:param:orte_rsh_agent:synonym:name:plm_rsh_agent
mca:plm:base:param:plm_rsh_agent:synonym_of:name:orte_rsh_agent

--
Samuel K. Gutierrez
Los Alamos National Laboratory


On Nov 5, 2010, at 12:41 PM, Joshua Bernstein wrote:


Hello All,

When building the examples included with OpenMPI version 1.5 I see a  
message printed as follows:


--
A deprecated MCA parameter value was specified in an MCA parameter
file.  Deprecated MCA parameters should be avoided; they may disappear
in future releases.

 Deprecated parameter: plm_rsh_agent
--

While I know that in pre 1.3.x releases the variable was  
pls_rsh_agent, plm_rsh_agent worked all the way through at least  
1.4.3. What is the new keyword name? I can't seem to find it in the  
FAQ located here:


http://www.open-mpi.org/faq/?category=rsh

-Josh
___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [OMPI users] Deprecated parameter: plm_rsh_agent

[Joshua Bernstein]

Thanks Samuel,

I should have checked ompi_info myself.

The FAQ on the website should probably be updated to reflect this 
function change.


-Joshua Bernstein
Software Development Manager
Penguin Computing

Samuel K. Gutierrez wrote:

Hi Josh,

I -think- the new name is orte_rsh_agent.  At least according to ompi_info.

$ ompi_info -a --parsable | grep orte_rsh_agent
mca:orte:base:param:orte_rsh_agent:value:ssh : rsh
mca:orte:base:param:orte_rsh_agent:data_source:default value
mca:orte:base:param:orte_rsh_agent:status:writable
mca:orte:base:param:orte_rsh_agent:help:The command used to launch 
executables on remote nodes (typically either "ssh" or "rsh")

mca:orte:base:param:orte_rsh_agent:deprecated:no
mca:orte:base:param:orte_rsh_agent:synonym:name:pls_rsh_agent
mca:orte:base:param:orte_rsh_agent:synonym:name:plm_rsh_agent
mca:plm:base:param:plm_rsh_agent:synonym_of:name:orte_rsh_agent

--
Samuel K. Gutierrez
Los Alamos National Laboratory


On Nov 5, 2010, at 12:41 PM, Joshua Bernstein wrote:


Hello All,

When building the examples included with OpenMPI version 1.5 I see a 
message printed as follows:


-- 


A deprecated MCA parameter value was specified in an MCA parameter
file.  Deprecated MCA parameters should be avoided; they may disappear
in future releases.

 Deprecated parameter: plm_rsh_agent
-- 



While I know that in pre 1.3.x releases the variable was 
pls_rsh_agent, plm_rsh_agent worked all the way through at least 
1.4.3. What is the new keyword name? I can't seem to find it in the 
FAQ located here:


http://www.open-mpi.org/faq/?category=rsh

-Josh
___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users


___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [OMPI users] Open MPI data transfer error

[Jack Bryan]

Thanks,
But, my code is too long to be posted. 
dozens of files, thousands of lines. 
Do you have better ideas ? 
Any help is appreciated. 
Jack
Nov. 5 2010
From: solarbik...@gmail.com
List-Post: users@lists.open-mpi.org
Date: Fri, 5 Nov 2010 11:20:57 -0700
To: us...@open-mpi.org
Subject: Re: [OMPI users] Open MPI data transfer error

As Prentice said, we can't help you without seeing your code.  openMPI has 
stood many trials from many programmers, with many bugs ironed out. So 
typically it is unlikely openMPI is the source of your error.  Without seeing 
your code the only logical conclusion is that something is wrong with your 
programming. 



On Fri, Nov 5, 2010 at 10:52 AM, Prentice Bisbal  wrote:


We can't help you with your coding problem without seeing your code.





Jack Bryan wrote:

> Thanks,

> I have used "cout" in c++ to print the values of data.

>

> The sender sends correct data to correct receiver.

>

> But, receiver gets wrong data from correct sender.

>

> why ?

>

> thanks

>

> Nov. 5 2010

>

>> Date: Fri, 5 Nov 2010 08:54:22 -0400

>> From: prent...@ias.edu

>> To: us...@open-mpi.org

>> Subject: Re: [OMPI users] Open MPI data transfer error

>>

>> Jack Bryan wrote:

>> >

>> > Hi,

>> >

>> > In my Open MPI program, one master sends data to 3 workers.

>> >

>> > Two workers can receive their data.

>> >

>> > But, the third worker can not get their data.

>> >

>> > Before sending data, the master sends a head information to each worker

>> > receiver

>> > so that each worker knows what the following data package is. (such as

>> > length, package tag).

>> >

>> > The third worker can get its head information message from master but

>> > cannot get its correct

>> > data package.

>> >

>> > It got the data that should be received by first worker, which get its

>> > correct data.

>> >

>>

>>

>> Jack,

>>

>> Providing the relevant sections of code here would be very helpful.

>>

>> 

>> I would tell you to add some printf statements to your code to see what

>> data is stored in your variables on the master before it sends them to

>> each node, but Jeff Squyres and I agreed to disagree in a civil manner

>> on that debugging technique earlier this week, and I'd hate to re-open

>> those old wounds by suggesting that technique here. ;)

>> 

>>

>> --

>> Prentice

___

users mailing list

us...@open-mpi.org

http://www.open-mpi.org/mailman/listinfo.cgi/users



-- 
David Zhang
University of California, San Diego


___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users  
  

Re: [OMPI users] Open MPI data transfer error

[Prentice Bisbal]

Choose one

A) Post only the relevant sections of the code. If you have syntax
error, it should be in the Send and Receive calls, or one of the lines
where the data is copied or read from the array/buffer/whatever that
you're sending or receiving.

B) Try reproducing your problem in a toy program that has only enough
code to reproduce your problem. For example, create an array, populate
it with data, send it, and then on the receiving end, receive it, and
print it out. Something simple like that. I find when I do that, I
usually find the error in my code.

Prentice


Jack Bryan wrote:
> Thanks,
> 
> But, my code is too long to be posted. 
> 
> dozens of files, thousands of lines. 
> 
> Do you have better ideas ? 
> 
> Any help is appreciated. 
> 
> Jack
> 
> Nov. 5 2010
> 
> From: solarbik...@gmail.com
> Date: Fri, 5 Nov 2010 11:20:57 -0700
> To: us...@open-mpi.org
> Subject: Re: [OMPI users] Open MPI data transfer error
> 
> As Prentice said, we can't help you without seeing your code.  openMPI
> has stood many trials from many programmers, with many bugs ironed out.
> So typically it is unlikely openMPI is the source of your error. 
> Without seeing your code the only logical conclusion is that something
> is wrong with your programming.
> 
> On Fri, Nov 5, 2010 at 10:52 AM, Prentice Bisbal  > wrote:
> 
> We can't help you with your coding problem without seeing your code.
> 
> 
> Jack Bryan wrote:
> > Thanks,
> > I have used "cout" in c++ to print the values of data.
> >
> > The sender sends correct data to correct receiver.
> >
> > But, receiver gets wrong data from correct sender.
> >
> > why ?
> >
> > thanks
> >
> > Nov. 5 2010
> >
> >> Date: Fri, 5 Nov 2010 08:54:22 -0400
> >> From: prent...@ias.edu 
> >> To: us...@open-mpi.org 
> >> Subject: Re: [OMPI users] Open MPI data transfer error
> >>
> >> Jack Bryan wrote:
> >> >
> >> > Hi,
> >> >
> >> > In my Open MPI program, one master sends data to 3 workers.
> >> >
> >> > Two workers can receive their data.
> >> >
> >> > But, the third worker can not get their data.
> >> >
> >> > Before sending data, the master sends a head information to
> each worker
> >> > receiver
> >> > so that each worker knows what the following data package is.
> (such as
> >> > length, package tag).
> >> >
> >> > The third worker can get its head information message from
> master but
> >> > cannot get its correct
> >> > data package.
> >> >
> >> > It got the data that should be received by first worker, which
> get its
> >> > correct data.
> >> >
> >>
> >>
> >> Jack,
> >>
> >> Providing the relevant sections of code here would be very helpful.
> >>
> >> 
> >> I would tell you to add some printf statements to your code to
> see what
> >> data is stored in your variables on the master before it sends
> them to
> >> each node, but Jeff Squyres and I agreed to disagree in a civil
> manner
> >> on that debugging technique earlier this week, and I'd hate to
> re-open
> >> those old wounds by suggesting that technique here. ;)
> >> 
> >>
> >> --
> >> Prentice
> ___
> users mailing list
> us...@open-mpi.org 
> http://www.open-mpi.org/mailman/listinfo.cgi/users
> 
> 
> 
> 
> -- 
> David Zhang
> University of California, San Diego
> 
> ___ users mailing list
> us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users
> 
> 
> 
> 
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users

-- 
Prentice Bisbal
Linux Software Support Specialist/System Administrator
School of Natural Sciences
Institute for Advanced Study
Princeton, NJ

Re: [OMPI users] Open MPI data transfer error

[Eugene Loh]

Debugging is not a straightforward task.  Even posting the code doesn't 
necessarily help (since no one may be motivated to help or they can't 
reproduce the problem or...).  You'll just have to try different things 
and see what works for you.  Another option is to trace the MPI calls.  
If a process sends a message, dump out the MPI_Send() arguments.  When a 
receiver receives, correspondingly dump those arguments.  Etc.  This 
might be a way of seeing what the program is doing in terms of MPI and 
thereby getting to suggestion B below.


How do you trace and sort through the resulting data?  That's another 
tough question.  Among other things, if you can't find a tool that fits 
your needs, you can use the PMPI layer to write wrappers.  Writing 
wrappers is like inserting printf() statements, but doesn't quite have 
the same amount of moral shame associated with it!


Prentice Bisbal wrote:


Choose one

A) Post only the relevant sections of the code. If you have syntax
error, it should be in the Send and Receive calls, or one of the lines
where the data is copied or read from the array/buffer/whatever that
you're sending or receiving.

B) Try reproducing your problem in a toy program that has only enough
code to reproduce your problem. For example, create an array, populate
it with data, send it, and then on the receiving end, receive it, and
print it out. Something simple like that. I find when I do that, I
usually find the error in my code.

Jack Bryan wrote:
 

But, my code is too long to be posted. 
dozens of files, thousands of lines. 
Do you have better ideas ? 
Any help is appreciated. 


Nov. 5 2010

From: solarbik...@gmail.com
Date: Fri, 5 Nov 2010 11:20:57 -0700
To: us...@open-mpi.org
Subject: Re: [OMPI users] Open MPI data transfer error

As Prentice said, we can't help you without seeing your code.  openMPI
has stood many trials from many programmers, with many bugs ironed out.
So typically it is unlikely openMPI is the source of your error. 
Without seeing your code the only logical conclusion is that something

is wrong with your programming.

On Fri, Nov 5, 2010 at 10:52 AM, Prentice Bisbal mailto:prent...@ias.edu>> wrote:

   We can't help you with your coding problem without seeing your code.


   Jack Bryan wrote:
   > Thanks,
   > I have used "cout" in c++ to print the values of data.
   >
   > The sender sends correct data to correct receiver.
   >
   > But, receiver gets wrong data from correct sender.
   >
   > why ?
   >
   > thanks
   >
   > Nov. 5 2010
   >
   >> Date: Fri, 5 Nov 2010 08:54:22 -0400
   >> From: prent...@ias.edu 
   >> To: us...@open-mpi.org 
   >> Subject: Re: [OMPI users] Open MPI data transfer error
   >>
   >> Jack Bryan wrote:
   >> >
   >> > Hi,
   >> >
   >> > In my Open MPI program, one master sends data to 3 workers.
   >> >
   >> > Two workers can receive their data.
   >> >
   >> > But, the third worker can not get their data.
   >> >
   >> > Before sending data, the master sends a head information to
   each worker
   >> > receiver
   >> > so that each worker knows what the following data package is.
   (such as
   >> > length, package tag).
   >> >
   >> > The third worker can get its head information message from
   master but
   >> > cannot get its correct
   >> > data package.
   >> >
   >> > It got the data that should be received by first worker, which
   get its
   >> > correct data.
   >> >
   >>
   >>
   >> Jack,
   >>
   >> Providing the relevant sections of code here would be very helpful.
   >>
   >> 
   >> I would tell you to add some printf statements to your code to
   see what
   >> data is stored in your variables on the master before it sends
   them to
   >> each node, but Jeff Squyres and I agreed to disagree in a civil
   manner
   >> on that debugging technique earlier this week, and I'd hate to
   re-open
   >> those old wounds by suggesting that technique here. ;)
   >>