[OMPI users] unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_*
Hi, OpenMPI 1.5.4 compiled with gcc 4.6.1 and linked with target app gives a load of linker messages like this one: /usr/bin/ld: ../../lib/libutil.a(parallel_utilities.o)(.debug_info+0x529d): unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_argv_null_ There are a lot of similar messages about other mpi_fortran_ symbols. Is it a known issue? Thanks, - D.
Re: [OMPI users] unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_*
Same error when configured with --with-pic --with-gnu-ld 2011/9/22 Dmitry N. Mikushin : > Hi, > > OpenMPI 1.5.4 compiled with gcc 4.6.1 and linked with target app gives > a load of linker messages like this one: > > /usr/bin/ld: ../../lib/libutil.a(parallel_utilities.o)(.debug_info+0x529d): > unresolvable R_X86_64_64 relocation against symbol > `mpi_fortran_argv_null_ > > There are a lot of similar messages about other mpi_fortran_ symbols. > Is it a known issue? > > Thanks, > - D. >
Re: [OMPI users] unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_*
This usually means that you're mixing compiler/linker flags somehow (e.g., built something with 32 bit, built something else with 64 bit, try to link them together). Can you verify that everything was built with all the same 32/64? On Sep 22, 2011, at 1:21 PM, Dmitry N. Mikushin wrote: > Hi, > > OpenMPI 1.5.4 compiled with gcc 4.6.1 and linked with target app gives > a load of linker messages like this one: > > /usr/bin/ld: ../../lib/libutil.a(parallel_utilities.o)(.debug_info+0x529d): > unresolvable R_X86_64_64 relocation against symbol > `mpi_fortran_argv_null_ > > There are a lot of similar messages about other mpi_fortran_ symbols. > Is it a known issue? > > Thanks, > - D. > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI users] unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_*
Hi Jeff, You're right because I also tried 1.4.3, and it's the same issue there. But what could be wrong? I'm using the simplest form - ../configure --prefix=/opt/openmpi_gcc-1.4.3/ and only installed compilers are system-default gcc and gfortran 4.6.1. Distro is ubuntu 11.10. There is no any mpi installed from packages, and no -m32 options around. What else could be the source? Thanks, - D. 2011/9/22 Jeff Squyres : > This usually means that you're mixing compiler/linker flags somehow (e.g., > built something with 32 bit, built something else with 64 bit, try to link > them together). > > Can you verify that everything was built with all the same 32/64? > > > On Sep 22, 2011, at 1:21 PM, Dmitry N. Mikushin wrote: > >> Hi, >> >> OpenMPI 1.5.4 compiled with gcc 4.6.1 and linked with target app gives >> a load of linker messages like this one: >> >> /usr/bin/ld: ../../lib/libutil.a(parallel_utilities.o)(.debug_info+0x529d): >> unresolvable R_X86_64_64 relocation against symbol >> `mpi_fortran_argv_null_ >> >> There are a lot of similar messages about other mpi_fortran_ symbols. >> Is it a known issue? >> >> Thanks, >> - D. >> ___ >> users mailing list >> us...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/users > > > -- > Jeff Squyres > jsquy...@cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users >
[OMPI users] openmpi -cc= option
I have openmpi via the distribution package: openmpi-1.4.3-7.fc14.i686 When attempting to make mpicc use another compiler as follows: mpicc -cc=icc ... I get gcc: unrecognized option '-cc=icc' /tmp/ccHd339C.o: In function `main': It looks like it's just passing this as option to gcc instead of using the supplied compiler. -- Uday
Re: [OMPI users] openmpi -cc= option
Right: -cc is not an option to Open MPI's mpicc wrapper compiler (I don't think it ever has been). See http://www.open-mpi.org/faq/?category=mpi-apps#cant-use-wrappers and http://www.open-mpi.org/faq/?category=mpi-apps#override-wrappers-after-v1.0. On Sep 22, 2011, at 3:44 PM, Uday Kumar Reddy B wrote: > I have openmpi via the distribution package: openmpi-1.4.3-7.fc14.i686 > > When attempting to make mpicc use another compiler as follows: > > mpicc -cc=icc ... > > I get > > gcc: unrecognized option '-cc=icc' > /tmp/ccHd339C.o: In function `main': > > It looks like it's just passing this as option to gcc instead of using > the supplied compiler. > > -- Uday > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI users] openmpi -cc= option
On Fri, Sep 23, 2011 at 1:21 AM, Jeff Squyres wrote:
> Right: -cc is not an option to Open MPI's mpicc wrapper compiler (I don't
> think it ever has been).
But that's not what the manual says ('man mpicc').
--
COMMAND LINE ARGUMENTS
-show - Show the commands that would be used without runnning them
-help - Give short help
-cc=name
- Use compiler name instead of the default choice.
Use this only if the compiler is compatible
with the MPICH library (see below)
---
-- Uday
>
> See http://www.open-mpi.org/faq/?category=mpi-apps#cant-use-wrappers and
> http://www.open-mpi.org/faq/?category=mpi-apps#override-wrappers-after-v1.0.
>
>
> On Sep 22, 2011, at 3:44 PM, Uday Kumar Reddy B wrote:
>
>> I have openmpi via the distribution package: openmpi-1.4.3-7.fc14.i686
>>
>> When attempting to make mpicc use another compiler as follows:
>>
>> mpicc -cc=icc ...
>>
>> I get
>>
>> gcc: unrecognized option '-cc=icc'
>> /tmp/ccHd339C.o: In function `main':
>>
>> It looks like it's just passing this as option to gcc instead of using
>> the supplied compiler.
>>
>> -- Uday
>> ___
>> users mailing list
>> us...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
> --
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
>
>
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
Re: [OMPI users] openmpi -cc= option
Ummm...yes, because you are getting the man page for the MPICH mpicc, not ours.
Try setting your manpage path to point to the OMPI install directory.
On Sep 22, 2011, at 1:55 PM, Uday Kumar Reddy B wrote:
> On Fri, Sep 23, 2011 at 1:21 AM, Jeff Squyres wrote:
>> Right: -cc is not an option to Open MPI's mpicc wrapper compiler (I don't
>> think it ever has been).
>
> But that's not what the manual says ('man mpicc').
>
> --
> COMMAND LINE ARGUMENTS
> -show - Show the commands that would be used without runnning them
> -help - Give short help
> -cc=name
> - Use compiler name instead of the default choice.
> Use this only if the compiler is compatible
> with the MPICH library (see below)
> ---
>
>
> -- Uday
>
>
>>
>> See http://www.open-mpi.org/faq/?category=mpi-apps#cant-use-wrappers and
>> http://www.open-mpi.org/faq/?category=mpi-apps#override-wrappers-after-v1.0.
>>
>>
>> On Sep 22, 2011, at 3:44 PM, Uday Kumar Reddy B wrote:
>>
>>> I have openmpi via the distribution package: openmpi-1.4.3-7.fc14.i686
>>>
>>> When attempting to make mpicc use another compiler as follows:
>>>
>>> mpicc -cc=icc ...
>>>
>>> I get
>>>
>>> gcc: unrecognized option '-cc=icc'
>>> /tmp/ccHd339C.o: In function `main':
>>>
>>> It looks like it's just passing this as option to gcc instead of using
>>> the supplied compiler.
>>>
>>> -- Uday
>>> ___
>>> users mailing list
>>> us...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>>
>> --
>> Jeff Squyres
>> jsquy...@cisco.com
>> For corporate legal information go to:
>> http://www.cisco.com/web/about/doing_business/legal/cri/
>>
>>
>> ___
>> users mailing list
>> us...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
Re: [OMPI users] openmpi -cc= option
On 09/23/2011 01:33 AM, Ralph Castain wrote:
Ummm...yes, because you are getting the man page for the MPICH mpicc, not ours.
Try setting your manpage path to point to the OMPI install directory.
Yes, but I can't find anything about -cc in openmpi's mpicc man page.
So, -cc should either not be supported or work as per mpich's mpicc if
you are wrapping around it.
Thanks,
-- Uday
On Sep 22, 2011, at 1:55 PM, Uday Kumar Reddy B wrote:
On Fri, Sep 23, 2011 at 1:21 AM, Jeff Squyres wrote:
Right: -cc is not an option to Open MPI's mpicc wrapper compiler (I don't think
it ever has been).
But that's not what the manual says ('man mpicc').
--
COMMAND LINE ARGUMENTS
-show - Show the commands that would be used without runnning them
-help - Give short help
-cc=name
- Use compiler name instead of the default choice.
Use this only if the compiler is compatible
with the MPICH library (see below)
---
-- Uday
See http://www.open-mpi.org/faq/?category=mpi-apps#cant-use-wrappers and
http://www.open-mpi.org/faq/?category=mpi-apps#override-wrappers-after-v1.0.
On Sep 22, 2011, at 3:44 PM, Uday Kumar Reddy B wrote:
I have openmpi via the distribution package: openmpi-1.4.3-7.fc14.i686
When attempting to make mpicc use another compiler as follows:
mpicc -cc=icc ...
I get
gcc: unrecognized option '-cc=icc'
/tmp/ccHd339C.o: In function `main':
It looks like it's just passing this as option to gcc instead of using
the supplied compiler.
-- Uday
___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
--
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/
___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
Re: [OMPI users] openmpi -cc= option
On Sep 22, 2011, at 2:17 PM, Uday Kumar Reddy B wrote:
>
>
> On 09/23/2011 01:33 AM, Ralph Castain wrote:
>> Ummm...yes, because you are getting the man page for the MPICH mpicc, not
>> ours. Try setting your manpage path to point to the OMPI install directory.
>
> Yes, but I can't find anything about -cc in openmpi's mpicc man page. So, -cc
> should either not be supported or work as per mpich's mpicc if you are
> wrapping around it.
As Jeff already said, OMPI doesn't support -cc, which is why it isn't mentioned
on the OMPI mpicc man page.
>
> Thanks,
>
> -- Uday
>
>>
>>
>> On Sep 22, 2011, at 1:55 PM, Uday Kumar Reddy B wrote:
>>
>>> On Fri, Sep 23, 2011 at 1:21 AM, Jeff Squyres wrote:
Right: -cc is not an option to Open MPI's mpicc wrapper compiler (I don't
think it ever has been).
>>>
>>> But that's not what the manual says ('man mpicc').
>>>
>>> --
>>> COMMAND LINE ARGUMENTS
>>> -show - Show the commands that would be used without runnning them
>>> -help - Give short help
>>> -cc=name
>>> - Use compiler name instead of the default choice.
>>> Use this only if the compiler is compatible
>>> with the MPICH library (see below)
>>> ---
>>>
>>>
>>> -- Uday
>>>
>>>
See http://www.open-mpi.org/faq/?category=mpi-apps#cant-use-wrappers and
http://www.open-mpi.org/faq/?category=mpi-apps#override-wrappers-after-v1.0.
On Sep 22, 2011, at 3:44 PM, Uday Kumar Reddy B wrote:
> I have openmpi via the distribution package: openmpi-1.4.3-7.fc14.i686
>
> When attempting to make mpicc use another compiler as follows:
>
> mpicc -cc=icc ...
>
> I get
>
> gcc: unrecognized option '-cc=icc'
> /tmp/ccHd339C.o: In function `main':
>
> It looks like it's just passing this as option to gcc instead of using
> the supplied compiler.
>
> -- Uday
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
--
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/
___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> ___
>>> users mailing list
>>> us...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>>
>> ___
>> users mailing list
>> us...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
Re: [OMPI users] openmpi -cc= option
On Sep 22, 2011, at 4:17 PM, Uday Kumar Reddy B wrote: > Yes, but I can't find anything about -cc in openmpi's mpicc man page. So, -cc > should either not be supported or work as per mpich's mpicc if you are > wrapping around it. mpicc has no way of knowing that an option is not supposed to work. It only knows about a small number of options that it can understand (e.g., --showme). Any other command line tokens are passed through to the underlying compiler. More specifically: how is mpicc supposed to know that any given option was intended for mpicc and not the underlying compiler, particularly the ones that it doesn't support? -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI users] openmpi -cc= option
On 09/23/2011 01:49 AM, Jeff Squyres wrote: On Sep 22, 2011, at 4:17 PM, Uday Kumar Reddy B wrote: Yes, but I can't find anything about -cc in openmpi's mpicc man page. So, -cc should either not be supported or work as per mpich's mpicc if you are wrapping around it. mpicc has no way of knowing that an option is not supposed to work. It only knows about a small number of options that it can understand (e.g., --showme). Any other command line tokens are passed through to the underlying compiler. More specifically: how is mpicc supposed to know that any given option was intended for mpicc and not the underlying compiler, particularly the ones that it doesn't support? Yes, it won't in the general case, but since -cc is accepted by mpich, you can as well assume it is not intended for the underlying compiler. If a user is indeed trying to pass -cc to some unknown compiler, the code is anyway not going to work with MPICH and probably most other MPIs. In any case, for portability purposes, shouldn't you support -cc? Or one could end up with distributions that compile on some or don't on another. If you indeed would not like to support it, it's better to check for -cc and throw an error than compile with a compiler user didn't intend to - the latter may go unnoticed. -- Uday
Re: [OMPI users] openmpi -cc= option
Jeff Squyres wrote: On Sep 22, 2011, at 4:17 PM, Uday Kumar Reddy B wrote: Yes, but I can't find anything about -cc in openmpi's mpicc man page. So, -cc should either not be supported or work as per mpich's mpicc if you are wrapping around it. mpicc has no way of knowing that an option is not supposed to work. It only knows about a small number of options that it can understand (e.g., --showme). Any other command line tokens are passed through to the underlying compiler. More specifically: how is mpicc supposed to know that any given option was intended for mpicc and not the underlying compiler, particularly the ones that it doesn't support? Uday If you want to use icc (instead of gcc or whatever is under your OpenMPI), just download the OpenMPI tarball to say /home/uday/install/openmpi, untar it, and build it with icc [and I would guess, with icpc and ifort as well]. Configure it with --prefix=/your/favorite/directory (e.g. something under your home directory, and pass the compiler names to it CC=icc, CXX=icpc, F77=ifort, FC=ifort. Use the prefix so as not to mix it with other MPI flavors that may be there. Building takes three steps: 1) configure; 2) make; 3) make install. Then set your PATH and LD_LIBRARY_PATH to point to the bin and lib subdirectories of /your/favorite/directory. In case of doubts, read the README file and the OpenMPI FAQ. You're good to go. That one will have icc under mpicc's hood, which I guess is what you want. It is that simple. My two cents, Gus Correa
Re: [OMPI users] openmpi -cc= option
On Sep 22, 2011, at 4:39 PM, Uday Kumar Reddy B wrote: >> More specifically: how is mpicc supposed to know that any given option was >> intended for mpicc and not the underlying compiler, particularly the ones >> that it doesn't support? > > Yes, it won't in the general case, but since -cc is accepted by mpich, you > can as well assume it is not intended for the underlying compiler. If a user > is indeed trying to pass -cc to some unknown compiler, the code is anyway not > going to work with MPICH and probably most other MPIs. In any case, for > portability purposes, shouldn't you support -cc? That would mean that we have to support all the CLI options for MPICH's mpicc. And they would likewise need to support ours. And we should also support Platform MPI's mpicc options. And ... It's a slippery slope, and we're not really willing to go down it. The real problem is that wrapper compilers are not standardized. Cray doesn't have one, for example (IIRC). And IBM AIX MPI doesn't either (also IIRC -- could be wrong on both of these). Users are unfortunately left in a lurch, and have to work around these issues in their application build systems. :-( > Or one could end up with distributions that compile on some or don't on > another. If you indeed would not like to support it, it's better to check for > -cc and throw an error than compile with a compiler user didn't intend to - > the latter may go unnoticed. What's the advantage to having mpicc throw the error vs. the underlying compiler? -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
[OMPI users] maximum size for read buffer in MPI_File_read/write
Hello,
MPI_File_read/write functions uses an integer to specify the size of
the buffer, for instance:
int MPI_File_read(MPI_File fh, void *buf, int count, MPI_Datatype
datatype, MPI_Status *status)
with:
count Number of elements in buffer (integer).
datatype Data type of each buffer element (handle).
However, using the maximum value of 32 bytes integers:
count = 2^31-1 = 2147483647 (and datatype = MPI_BYTE)
MPI_file_read only reads 2^31-2^12 = 2147479552 bytes.
This means that 4095 bytes are ignored.
I am not aware of this specific limit for integers in (Open) MPI
function calls. Is this supposed to be correct?
MPI_File_read/write does not return an error (but MPI_Get_count states
that only 2147479552 bytes are considered). Find attached a C++ code
example which tries to write and read 2^31-1 bytes.
I am using Open MPI 1.4.2 compiled with the Intel compiler.
Best regards,
German
#include
#include
#include
#include
#ifndef NO_MPI
#include
#endif
#include
using namespace std;
int main( int nargs, char *args[] )
{
int mpi_rank, mpi_size;
#ifndef NO_MPI
MPI_Init( &nargs, &args );
MPI_Comm_rank( MPI_COMM_WORLD, &mpi_rank );
MPI_Comm_size( MPI_COMM_WORLD, &mpi_size );
#else
mpi_rank = 0;
mpi_size = 1;
#endif
int nbuf = std::numeric_limits::max();
if ( nargs < 2 ){
if ( mpi_rank == 0 ){
cerr<<"usage:\n"
<<"--\n"
<
[OMPI users] on cluster job slowdown near end
Hi, A job I am running slows down as it approaches the end. I'd appreciate any ideas you may have on possible cause or what else I can look at for diagnostic info. Environment: * Linux cluster, very recent version of Fedora. * openmpi 1.5 Characteristics of job: * Tasks are all the same size and duration. * 56K tasks, but multiple tasks given to each process. * Typically run 120 processes. * Slowdown starts at ~52K completed, then rate of completion of each task declines geometrically from ~1k/minute to 4/minute at 54K. Here are some queries done when the slowdown occurs: * "ps" on master node - most processes in suspend state: F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD 0 S 3348 27933 15675 0 80 0 - 13608 poll_s pts/000:00:00 mpiexec 0 S 3348 28009 27933 14 80 0 - 227632 epoll_ pts/0 00:08:13 C5MPI 0 S 3348 28011 27933 14 80 0 - 227672 epoll_ pts/0 00:08:17 C5MPI 0 S 3348 28013 27933 13 80 0 - 227713 epoll_ pts/0 00:08:06 C5MPI 0 S 3348 28015 27933 13 80 0 - 227844 epoll_ pts/0 00:08:02 C5MPI 0 S 3348 28017 27933 14 80 0 - 227849 epoll_ pts/0 00:08:13 C5MPI 0 S 3348 28019 27933 13 80 0 - 227892 epoll_ pts/0 00:08:07 C5MPI * file handles (allocated handle count is ~constant): $ cat /proc/sys/fs/file-nr 39680 801014 * Processes in a suspend or run state (varies): $ orte-top -pid 27933 | grep ' S |' | wc -l 124 $ orte-top -pid 27933 | grep ' R |' Rank | Nodename | Command | Pid | State | Time | Pri | #threads | Vsize |RSS | Peak Vsize | Shr Size | 0 | rubel-001 | C5MPI | 14700 | R | 2.2H | 20 |1 | 246208 | 12660 | 246208 |17664 | 1 | rubel-001 | C5MPI | 14702 | R | 2.2H | 20 |1 | 245360 | 44860 | 245360 |17664 |
[OMPI users] PATH settings
Hi, I am trying to figure out how openmpi (1.4.3) sets its PATH for executables. From the man page: Locating Files If no relative or absolute path is specified for a file, Open MPI will first look for files by searching the directories specified by the --path option. If there is no --path option set or if the file is not found at the --path location, then Open MPI will search the user’s PATH environment variable as defined on the source node(s). This does not appear to be entirely correct - as far as I can tell openmpi always prepends its own bin directory to the PATH before searching for the executable. Can that be switched off? Furthermore, openmpi appears to use a) the current value of PATH on the node where mpiexec is running; b) whatever PATH is used by ssh on the remote nodes. Correct? Thus, export PATH=/path/to/special/bin:$PATH mpiexec -n 2 -H n1,n2 special (n1 being the local node) will usually fail even if the directory structure is identical on the two nodes. For this to work mpiexec -n 2 -H n1,n2 -x PATH special is necessary. What I would like to see is a configure option that allows me to configure openmpi such that the current PATH on the node where mpiexec is running is used as the PATH on all nodes (by default). Or is there a reason why that is a really bad idea? Cheers, Martin -- Martin Siegert Head, Research Computing WestGrid/ComputeCanada Site Lead IT Servicesphone: 778 782-4691 Simon Fraser Universityfax: 778 782-4242 Burnaby, British Columbia email: sieg...@sfu.ca Canada V5A 1S6
