[OMPI users] RDMA-CM

2008-06-17 Thread Neeraj Chourasia
Hello everyone,    I downloaded openmpi-1.3 version from night tarballs to check RDMA-CM support. I am able to compile and install it, but dont know how to run it as there is no documentation provided. Did someone try running it with OpenMPI?My another question is Does OpenMPI1.3 has progress-t

Re: [OMPI users] RDMA-CM

2008-06-17 Thread Jeff Squyres
On Jun 17, 2008, at 5:12 AM, Neeraj Chourasia wrote: I downloaded openmpi-1.3 version from night tarballs to check RDMA-CM support. I am able to compile and install it, but dont know how to run it as there is no documentation provided. Did someone try running it with OpenMPI? The them

Re: [OMPI users] gfortran bindings apparently not built on mac os leopard

2008-06-17 Thread Jeff Squyres
This is pretty odd. What files do you see in $prefix/share/openmpi/mpi*txt? What do you see from: grep required_file $prefix/share/openmpi/mpi*txt On Jun 16, 2008, at 2:12 PM, Weirs, V Gregory wrote: Dave Gunter: According to ./configure —help, by default —enable-mpi-f77 and — enabl

Re: [OMPI users] help me please, about Open MPI

2008-06-17 Thread Jeff Squyres
Is /tmp writable on your compute nodes? On Jun 16, 2008, at 1:49 PM, Tony Smith wrote: Dear Sir: thanks. I have changed it to its absolute path: /ptmp/myname/openmpi123/ompi123_install/bin/mpirun -np 8 /ptmp/ myname/openmpi123/openmpi-1.2.3/examples/hello_c But I still got the error : ==

Re: [OMPI users] Roadrunner blasts past the petaflop mark with Open MPI

2008-06-17 Thread Brad Benton
On Mon, Jun 16, 2008 at 9:46 PM, Brock Palen wrote: > Brad just curious. > Did you tweak any other values for starting and running a job on such a > large system? You say unmodified, but OpenMPI lets you tweak many values > at runtime. > Ahh...sorry for the confusion. By "unmodified", I meant

Re: [OMPI users] Roadrunner blasts past the petaflop mark with Open MPI

2008-06-17 Thread Jeff Squyres
On Jun 17, 2008, at 10:18 AM, Brad Benton wrote: The main run-time parameters used are as follows: - For scalable process launch routed = binomial Note that this is a new parameter for OMPI v1.3 -- it has to do with how non-MPI communications are sent around (e.g., OMPI startup coordina

Re: [OMPI users] help me please, about Open MPI

2008-06-17 Thread Tony Smith
thanks, I changed /tmp and /ptmp and its sub directories to writable . But , the same errors. thanks, > From: jsquy...@cisco.com > To: us...@open-mpi.org > Date: Tue, 17 Jun 2008 09:10:18 -0400 > Subject: Re: [OMPI users] help me please, about Open MPI > > Is /tmp writable on your compute nod

Re: [OMPI users] help me please, about Open MPI

2008-06-17 Thread Jeff Squyres
What are the exact permissions on /tmp? They should likely be 1777. Do you have the TMPDIR or TMP environment variables set? If so, is that directory also world-writable? (if set, these will override the default location of /tmp) The error you are seeing (session directory failed) usuall

Re: [OMPI users] OpenMPI with bounds checking

2008-06-17 Thread Jeff Squyres
I'm afraid that we've never tested OMPI under these conditions; I don't know exactly what xgcc is testing for, so I don't know why it would fail. There could be weird interactions between the OFED stack and xgcc...? (i.e., memory "appears" from the kernel via ibv_* function calls, etc.)

Re: [OMPI users] OpenMPI with bounds checking

2008-06-17 Thread Gabriele Fatigati
Hi Jeff, i compiled OpenMPi using xgcc 4.0.2. I noted that the program stops before to do process spawn. Very strange. 2008/6/17 Jeff Squyres : > I'm afraid that we've never tested OMPI under these conditions; I don't > know exactly what xgcc is testing for, so I don't know why it would fail. >

[OMPI users] orted runs on host, but doesn't seem to communicate

2008-06-17 Thread jody
Hi Since i upgraded from open-mpi .2.2 to open-mpi 1.2.5 and had to reinstall my machine aim-plankton (fedora 8 instead of fedora 6) open-mpi doesn't work correctly anymore: When i start an application from aim-plankton to run on an other machine, it seems to hang (no output, not even from debug-d

Re: [OMPI users] Application Context and OpenMPI 1.2.4

2008-06-17 Thread Ralph H Castain
Hi Pat A friendly elf forwarded this to me, so please be sure to explicitly include me on any reply. Was that the only error message you received? I would have expected a trail of "error_log" outputs that would help me understand where this came from. If not, I can give you some debug flags to se

[OMPI users] btl parameter is not set to openib on node with ib card

2008-06-17 Thread SLIM H.A.
Hi OpenMPI does not pick up the infiniband component on our nodes with Mellanox cards: ompi_info --param btl openib returns MCA btl: parameter "btl_base_debug" (current value: "0") If btl_base_debug is 1 standard debug is output, if > 1 verbose debug is output MCA btl: parameter "btl"

Re: [OMPI users] btl parameter is not set to openib on node with ib card

2008-06-17 Thread Jeff Squyres
As you surmised, it looks like your "btl" parameter is being set somewhere, and it excludes the openib BTL. Check out this FAQ entry: http://www.open-mpi.org/faq/?category=tuning#setting-mca-params On Jun 17, 2008, at 10:49 AM, SLIM H.A. wrote: Hi OpenMPI does not pick up the infinib

[OMPI users] [SPAM:### 82%] RE: help me please, about Open MPI

2008-06-17 Thread Tony Smith
thanks, drwxrwxrwt9 root root102400 Jun 17 10:36 tmp I have no TMPDIR or TMP environment variables set . SO, do I need to set up some for them ? thanks a lot ! June 17 2008 > From: jsquy...@cisco.com > To: us...@open-mpi.org > Date: Tue, 17 Jun 2008 10:35:06 -0400 > Subject: Re: [

Re: [OMPI users] help me please, about Open MPI

2008-06-17 Thread Tony Smith
thanks, My configure : ./configure --prefix=/ptmp/myname/openmpi --enable-static --disable-shared CC=icc CXX=icpc F77=ifort FC=ifort --with-gm=/opt/gm -with-tm=/usr/spool/PBS/is that correct ? thanks > From: jsquy...@cisco.com > To: us...@open-mpi.org > Date: Tue, 17 Jun 2008 10:35:06 -040

Re: [OMPI users] help me please, about Open MPI

2008-06-17 Thread Jeff Squyres
Sure, that configure line should be fine. But that's a different issue than the permissions on your temp directories. The environment variables I was asking about are what are set at run time -- not when you configure/build OMPI. If you're using Torque, I *believe* that it sets TMPDIR in

Re: [OMPI users] gfortran bindings apparently not built on mac os leopard

2008-06-17 Thread Weirs, V Gregory
I have mpic++-wrapper-data.txt mpicc-wrapper-data.txt mpicxx-wrapper-data.txt mpif77-wrapper-data.txt mpif90-wrapper-data.txt And for the grep: mpic++-wrapper-data.txt:required_file= mpicc-wrapper-data.txt:required_file= mpicxx-wrapper-data.txt:required_file= mpif77-wrapper-data.txt:required_fi

Re: [OMPI users] btl parameter is not set to openib on node with ibcard

2008-06-17 Thread SLIM H.A.
Jeff, Thanks, I forgot I had a ~/.openmpi/mca-params.conf file laying around from a previous test. Henk > -Original Message- > From: users-boun...@open-mpi.org > [mailto:users-boun...@open-mpi.org] On Behalf Of Jeff Squyres > Sent: 17 June 2008 16:03 > To: Open MPI Users > Subject: Re:

Re: [OMPI users] Question about RDMA

2008-06-17 Thread Jeff Squyres
On Jun 6, 2008, at 6:03 AM, Gabriele Fatigati wrote: Hi Jeff, Sorry for the delay in replying -- I was on vacation all last week. thanks for you reply. I did understand previous questions about RDMA. Ever with SKaMPI, i tried to run with mpi_leave_pinned = 1, as you have suggested. But a

Re: [OMPI users] OpenMPI bug?

2008-06-17 Thread Jeff Squyres
Thanks for digging into this! The assembly portion of OMPI is quite squirrelly and dangerous to mess with. We'll need to check into this carefully to make sure that it works properly on all supported architectures... As for other bounds checking, would you mind checking the OMPI developm

Re: [OMPI users] SLURM and OpenMPI

2008-06-17 Thread Ralph H Castain
I can believe 1.2.x has problems in that regard. Some of that has nothing to do with slurm and reflects internal issues with 1.2. We have made it much more resistant to those problems in the upcoming 1.3 release, but there is no plan to retrofit those changes to 1.2. Part of the problem was that w

Re: [OMPI users] help me please, about Open MPI

2008-06-17 Thread Tony Smith
thanks, I added "setenv TMPDIR /tmp:$TMPDIR" in my job script file. so , echo $TMPDIR : /tmp:/local2/pbs/myname/37911.hpc-cluster but the same errors. thanks > From: jsquy...@cisco.com > To: us...@open-mpi.org > Date: Tue, 17 Jun 2008 11:55:05 -0400 > Subject: Re: [OMPI users] help me

Re: [OMPI users] help me please, about Open MPI

2008-06-17 Thread Ilmar Wilbers
TMPDIR should be a single directory: export TMPDIR=/tmp It should not be a list of directories seperated by a ':'. "setenv TMPDIR /tmp" (or "setenv TMPDIR /local2/pbs/myname/37911.hpc-cluster") would hence be the correct line. After setting the variable, try 'cd $TMPDIR' 'touch test' to veri

Re: [OMPI users] help me please, about Open MPI

2008-06-17 Thread Adrian Knoth
On Tue, Jun 17, 2008 at 01:11:15PM -0400, Tony Smith wrote: > so , echo $TMPDIR : > > /tmp:/local2/pbs/myname/37911.hpc-cluster > > but the same errors. Have you checked "ls -ld /tmp/"? Jeff has been asking you twice to provide the exact permissions, so I guess he would be delighted to ev

Re: [OMPI users] help me please, about Open MPI

2008-06-17 Thread Adrian Knoth
On Tue, Jun 17, 2008 at 07:30:27PM +0200, Adrian Knoth wrote: > Have you checked "ls -ld /tmp/"? Jeff has been asking you twice to Never mind, I just saw your other mail (tagged with [SPAM]) ;) -- Cluster and Metacomputing Working Group Friedrich-Schiller-Universität Jena, Germany private: ht

Re: [OMPI users] gfortran bindings apparently not built on mac os leopard

2008-06-17 Thread Weirs, V Gregory
Alright, I'm an idiot. I didn't adjust my DYLD_LIBRARY_PATH. Sorry about that- Greg On 6/17/08 7:08 AM, "Jeff Squyres" wrote: This is pretty odd. What files do you see in $prefix/share/openmpi/mpi*txt? What do you see from: grep required_file $prefix/share/openmpi/mpi*txt On Jun 16, 2

Re: [OMPI users] gfortran bindings apparently not built on mac os leopard

2008-06-17 Thread Jeff Squyres
Sorry for not replying earlier -- glad you figured it out. Yes, if you didn't also reset the LIBRARY_PATH, the "wrong" libmpi (and friends) will get picked up, and it will therefore use the hard- coded directory path in those "wrong" libraries to find the wrapper data text files (the ones wi